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1  Introduction  to  Data  Embedding 

[Ande98]  offers  two  reasons  for  the  rapid  growth  of  interest  in  data  embedding  techniques  since 
1996.  First,  an  appreciation  of  new  market  opportunities  created  by  digital  distribution  is 
coupled  with  a  fear  that  digital  works  are  too  easy  to  copy.  Traditional  art  can  be  authenticated 
by  studying,  for  example,  paint  brush  strokes.  On  the  other  hand,  digital  works  do  not  lend 
themselves  to  similar  types  of  analysis.  Because  of  this,  the  publishing  and  broadcasting 
industries  have  become  interested  in  techniques  for  embedding  copyright  marks  and  serial 
numbers  in  digital  films,  audio  recordings,  books,  and  multimedia  products.  Second,  policy 
moves  by  various  governments  to  restrict  the  availability  of  encryption  services  have  motivated 
people  to  study  methods  by  which  private  messages  can  be  hidden  in  seemingly  innocuous  cover 
messages.  The  current  basis  for  export  control  for  the  United  States  -  the  Wassenaar 
Arrangement  of  Export  Controls  for  Conventional  Arms  and  Dual-Use  Goods  and  Technologies 
(adopted  on  13  July  1996)  -  includes  cryptographic  products  on  its  export  control  list.  Although 
this  stance  has  been  softened  by  recent  executive  and  legislative  rulings  [McCu99,  Fran99],  it 
has  prompted  the  use  of  data  embedding  tools  to  conceal  information  from  casual  interception. 

The  field  of  data  embedding  has  its  infancy  in  steganography.  The  word  steganography  comes 
from  the  Greek  steganos  (covered  or  secret)  and  -graphia  (writing  or  drawing)  and  thus  means, 
literally,  covered  writing.  Steganography  serves  to  hide  secret  messages  in  other  messages  such 
that  the  existence  of  the  secret  message  is  concealed.  Historical  steganographic  techniques 
include  invisible  ink,  tiny  pin  punctures  on  selected  characters,  minute  differences  between 
handwritten  characters,  pencil  marks  on  typewritten  characters,  and  so  on  [Kahn96].  However, 
steganographic  techniques  do  not  have  to  be  technically  sophisticated  to  be  successful.  It  is 
common  to  hide  messages  in  the  first  letters  from  words  or  sentences  of  some  innocent  looking 
text.  In  addition,  one  could  write  a  letter  or  create  a  painting  that  appears  inconspicuous  to  an 
observer.  [Fran96]  gives  the  simple  following  example:  one  could  send  write  a  letter  to  his 
brother  containing  a  painting  from  his  three-year-old  daughter  —  the  painting  of  an  apple  tree.  A 
normal  observer,  like  a  policeman,  would  see  the  picture.  The  covering  letter  will  tell  him  that 
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the  proud  father  has  sent  the  painting  of  his  daughter.  However,  the  brother  would  count  the 
number  of  apples  on  the  picture  and  know:  "Our  meeting  will  be  at  3  pm." 

It  is  an  erroneous  to  equate  steganography  to  cryptography.  Cryptography  deals  with  the  process 
of  disguising  messages  such  that  unauthorized  persons  cannot  deduce  the  substance  of  the 
message.  Anyone  eavesdropping  on  the  communication  channel  will  be  able  to  see  the  exchange 
of  ciphertext  (encrypted  message),  but  without  the  secret  key,  will  not  be  able  to  retrieve  the 
original  plaintext  (intelligible  message).  Encryption  provides  confidentiality,  but  sometimes  this 
is  not  enough.  For  example,  the  detection  of  encrypted  message  traffic  between  a  military 
person  and  a  hostile  government  has  obvious  implications.  Steganography  is  the  art  and  science 
of  communicating  through  covert  channels.  Steganography  serves  to  hide  (secret)  messages  in 
other  messages  such  that  the  existence  of  the  (secret)  message  is  concealed.  In  the  simplest 
implementation,  the  sender  writes  an  innocuous  message  and  then  conceals  a  secret  message  on 
the  same  piece  of  paper. 

The  field  of  embedding  data  has  evolved  from  the  ancient  art  of  steganography.  Data  embedding 
(also  known  as  information  hiding)  informally  refers  to  "hidden  writing",  i.e.,  writing  that  is  not 
readily  discernible  to  the  casual  observer.  This  is  a  more  general  case  of  steganography.  In 
steganography,  the  presence  of  a  message  is  concealed.  This  is  not  a  requirement  for  more 
general  data  embedding  techniques. 

With  the  growing  ubiquitous  nature  of  the  Internet  and  of  electronic  correspondence, 
steganographic  techniques  naturally  evolved  to  use  digital  images,  digital  video,  or  digital  audio 
for  hiding  messages.  The  applications  for  these  techniques  have  widened  to  include  more  than 
covert  communications.  There  are  three  major  areas  of  data  embedding  applications.  These 
areas  are  differentiated  not  by  the  techniques  that  they  use,  but  by  the  relationship  between  the 
embedded  data  and  the  object  wherein  which  the  data  is  embedded  into.  These  areas  of 
applications  are  (1)  covert  communications  (or  steganography),  (2)  information  augmentation  (or 
captioning),  and  (3)  document  marking  (Figure  1).  Document  marking  can  be  further  subdivided 
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into  source  and  content  authentication.  In  covert  communications  applications,  the  embedded 
data  has  no  relationship  with  the  embedded  data.  Undetectability  of  the  embedded  data  is 
paramount.  In  captioning  applications,  the  embedded  data  provides  additional  information  about 
the  object  into  which  it  was  embedded.  Undetectability  is  not  a  concern,  although  capacity  is 
typically  maximized.  In  addition,  robustness  is  not  an  issue  since  there  is  no  motivation  for 
someone  to  remove  the  additional  information.  In  document  marking  for  source  authentication, 
the  embedded  data  provides  ownership  information.  Unlike  in  a  covert  communication  scenario, 
undetectability  and  capacity  is  not  significant.  It  is  understood  that  every  principal  with  access  to 
the  object  knows  that  it  has  been  marked.  As  such,  robustness  to  attempts  to  remove  the 
ownership  information  is  critical.  It  is  assumed,  especially  in  copyright  protection  scenarios,  that 
there  exists  an  unscrupulous  person  who  is  motivated  to  remove  embedded  ownership 
information.  There  is  a  limit  on  what  this  attacker  can  do  to  the  image,  however,  since  he  or  she 
also  wishes  to  preserve  the  quality  of  the  image.  In  a  content  authentication  application,  tamper 
detection  seals  are  embedded  into  the  image.  These  seals  "break"  whenever  the  content  of  the 
image  is  modified.  The  various  requirements  for  these  three  application  areas  will  be  further 
discussed  in  this  paper. 


Figure  1.  Application  Areas  of  Data  Embedding  Techniques. 

In  all  applications,  data  embedding  involves  imperceptibly  embedding  marks  or  data  into  a 
digital  media  object  to  enhance  or  protect  its  value.  The  embedded  data  may  contain  information 
such  as  authentication  or  copyright  codes.  Although  the  marked  media  is  perceptually 
unchanged,  the  embedding  method  imposes  detectable  or  extractable  modifications,  given  the 
appropriate  extraction  algorithm.  Ideally,  the  imposed  modification  does  not  degrade  the  host 
beyond  its  utility  value. 
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Although  most  digital  objects  in  the  digital  universe  can  be  processed  to  hide  additional 
information,  this  paper  deals  with  a  subset  of  these  digital  objects.  More  specifically,  this  paper 
deals  with  information  hiding  techniques  in  digital  images. 

Data  embedding  terminology  was  standardized  in  the  First  Information  Hiding  Workshop,  which 
was  held  in  the  United  Kingdom  in  1994.  Whenever  possible,  this  paper  will  use  the  same 
terminology.  The  data  that  is  embedded  into  the  digital  object  is  referred  to  as  the  embedded  data 
or  as  the  plain  object.  The  digital  object  into  it  is  embedded  into  is  considered  to  be  the  cover 
object.  Embedding  the  plain  object  into  the  cover  object  results  into  the  stego  or  marked  object 
(Figure  2).  A  stego-key  or  marking  key  is  normally  used  to  control  the  embedding  process  and  to 
restrict  the  recovery  of  the  embedded  data  to  principals  who  know  it.  Although  the  terms  stego 
object  and  stego-key  are  reminiscent  of  pure  steganographic  techniques,  in  this  paper,  they  are 
used  in  the  general  data  embedding  framework. 


Figure  2.  Data  Embedding  Terminology. 

2  Data  Embedding  Framework 

There  is  no  comprehensive  theory  of  steganography  in  the  same  marmer  Shannon  treated  the 
theory  of  encryption  [Shan48]  or  Simmons  considered  authentication  systems  [Simm83]. 
Recently,  however,  work  in  information  theoretic  is  being  conducted  for  the  data  embedding 
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communication  channel  [Cach98,  Mitt99].  Informally,  all  elements  that  comprise  the  data 
embedding  framework  can  be  illustrated  using  the  "Prisoner’s  Problem"  formulated  by  Simmons 
[Simm83]  in  1983'.  In  this  scenario,  two  principals  --  Alice  and  Bob  —  are  in  prison  and  they 
wish  to  hatch  an  escape  plan.  All  of  their  messages  pass  through  Willie,  the  warden.  If  Willie 
detects  any  encrypted  or  suspicious  message,  he  will  frustrate  their  plan.  Therefore,  Alice  and 
Bob  must  find  some  way  of  hiding  their  escape  plans  (plain  objects)  in  an  innocuous-looking 
exchange  of  public  messages  (cover  objects).  Suppose  the  two  prisoners  communicate  by 
sending  digital  images.  If  Alice  and  Bob  can  hide  their  (private)  messages  in  the  digital  images 
that  they  exchange  in  such  a  way  that  Willie  is  not  aware  of  the  private  communication,  they  can 
successfully  plan  their  escape.  If  Willie  manages  to  detect  the  presence  of  a  covert  message 
(through  statistical  analysis,  for  example),  Alice  and  Bob  are  severely  penalized.  Figure  3 
illustrates  the  elements  of  this  communication  fi-amework. 


Figure  3.  General  Data  Embedding  Framework. 


[Mitt99]  proposes  an  information-theoretic  approach  to  data  embedding  (Figure  4).  More 
formally,  a  data  embedding  system  is  used  to  transmit  (secret)  message  V  from  a  sender  (Alice) 
to  a  destination  (Bob)  in  such  a  way  that  an  intermediate  party  (Willie)  is  not  able  to  notice  that 


'  http://www.conceptlabs.co.uk/alicebob.html  has  a  humorous,  yet  accurate,  treatment  of  the 
Prisoner’s  Problem. 
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the  stego  object  X  contains  the  hidden  message^.  If  Willie  has  control  over  the  communication 
channel,  he  can  modify  a  suspect  message  X  and  transform  it  into  a  modified  version  Y.  If  such 
modification  attacks  might  occur,  the  data  embedding  system  should  be  robust  against  small 
distortions  in  the  sense  that  the  embedded  information  will  still  reach  Bob. 


Figure  4.  Formal  Model  for  a  Data  Embedding  Framework  [Mitt99]. 


In  the  data  embedding  scheme,  a  (secret)  message  V  is  embedded  in  some  cover  object.  The 
embedding  of  the  message  is  performed  by  the  data  embedding  encoder  which,  depending  on 
some  secret  key  K,  merges  the  message  V  into  the  cover  object  U.  For  each  key  value  k,  the 
stego  encoder  (.,.)  produces  the  stego  object  X  =  (U,  V) .  It  is  assumed  that  the  encoder  has 

an  encoder  inverse  (.,.) ,  i.e.,  (U,  V),U)  =  V .  The  stego  object  should  look  genuine,  i.e., 

the  stego  object  should  not  be  distinguishable  from  a  typical  message  of  the  message  source.  A 
possible  way  of  imposing  this  condition  mathematically  is  by  choosing  a  suitable  distortion 
measure  and  by  requiring  for  every  key  value  k,  the  encoder  constraint 

Ed(U,X)<D' 


^  It  is  important  to  note  that  Willie  does  not  "notice"  that  the  stego  object  contains  a  hidden 
message.  This  imposes  that  the  perceptual  quality  of  the  image  is  maintained.  Undetectability 
through  statistical  methods  may  or  may  not  be  a  requirement. 
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where  the  expectation  operation  E  is  with  respect  to  the  joint  probability  distribution  on  U  and  X. 
The  bound  is  chosen  suitably  small  to  guarantee  that  the  stego  object  X  is  essentially 
indistinguishable  from  the  cover  data  U. 

In  a  covert  communication  application,  the  steganographic  encoder  should  not  introduce  any 
statistically  detectable  artifacts  when  transforming  U  into  X.  In  digital  watermarking 
applications,  the  embedding  of  the  secret  message  should  be  robust  in  addition  to  satisfying  the 
encoder  constraint.  This  robustness  requirement  can  be  modeled  by  a  data  embedding  channel  in 
the  following  way.  The  attacker  is  allowed  to  modify  the  stego  object  only  in  a  limited  way, 
otherwise  the  quality  of  X  will  suffer  too  much  —  a  distorted  image  Y  after  a  distortion  attack 
must  still  have  reasonable  quality.  This  quality  requirement,  which  will  be  called  the  channel 
constraint,  can  be  expressed  as 


Ed{X,Y)<D' 

where  the  expectation  is  with  respect  to  the  joint  probability  distribution  on  X  and  Y,  This 
implies  that  the  distorted  image  Y  must  be  close  to  the  stego  image  X  for  small  D' .  The 
channel  constraint  can  also  be  generalized  to  include  geometric  transformations  by  the  attacker. 

It  is  implicitly  assumed  that  the  attacker  has  no  knowledge  of  the  secret  message  V,  the  secret 
key  K,  or  the  source  message  U  other  than  that  contained  in  her  observation  X.  For  instance,  if 
he  knew  U,  he  could  set  Y=U  and  deceive  the  receiver. 

The  goal  of  the  decoder  is  to  reconstruct  the  secret  message  V  from  the  received  (distorted) 
message  Y  using  the  secret  key  K.  In  many  cases,  it  is  assumed  that  the  decoder  has  no 
knowledge  of  the  cover  message  U  (oblivious  scheme).  However,  there  are  applications  where 
the  decoder  has  access  to  the  cover  message  (non-oblivious  scheme).  Knowledge  of  the  cover 
message  can  help  in  the  decoding  process.  One  can  utilize  the  cover  image  to  pre-process  the 
(distorted)  stego  object  before  invoking  the  decoding  processes.  In  any  case,  an  ideal  decoder 
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should  not  give  an  attacker  information  on  how  to  remove  the  embedded  data.  [Kalk,  Kalk97] 
has  shown  that  a  watermark  detector  —  if  not  properly  designed  —  can  be  used  to  remove  a  digital 
watermark. 

As  in  cryptography,  we  assume  that  the  security  should  depend  on  a  secret  key  that  Alice  and 
Bob  have  somehow  managed  to  share.  The  algorithm  for  the  data  embedding  encoder  is 
assumed  to  be  public  knowledge,  even  though  in  practice  this  would  be  rather  difficult,  if  not 
almost  impossible,  to  obtain  this  knowledge^.  In  a  watermarking  application,  security  also  rests 
on  the  inability  of  an  attacker  to  remove  the  embedded  watermark  or  render  it  unreadable 
without  degrading  the  quality  of  the  stego  object.  In  a  covert  communication  application,  any 
artifact  introduced  into  the  cover  object  by  the  steganographic  encoder  must  be  statistically 
undetectable. 

2.1  Data  Embedding  in  Digital  Images 

This  paper  deals  with  only  a  subset  of  digital  objects  that  can  be  used  as  a  container  for 
embedded  data.  More  specifically,  this  paper  deals  with  information  hiding  techniques  in  digital 
images.  In  its  raster  data  format,  digital  images  are  represented  by  a  two-dimensional  array  (e.g., 
a  matrix)  in  which  each  element  of  the  matrix  corresponds  to  a  single  pixel"*  in  the  displayed 
image.  Each  pixel  may  or  may  not  be  physically  square.  The  element  value  in  the  matrix  is 
usually  determined  by  the  type  of  the  image.  There  are  four  basic  types  of  images:  (1)  binary 
images,  (2)  intensity  images,  (3)  indexed  images,  and  (4)  RGB  images.  The  differences  of  these 
different  types  are  summarized  in  Table  1. 


^  This  corresponds  to  the  Kerkhoffs  principle  of  cryptography  [Kerkl883]. 

"*  The  term  "pixel"  is  derived  from  "picture  element."  Physically,  it  corresponds  to  a  single  dot 
on  a  computer  display. 
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Table  1.  Different  Types  of  Digital  Images. 


Type  of  Image 

Description 

Binary  Images 

Each  pixel  assumes  one  of  only  two  discrete  values:  on  (white)  and 
off  (black).  Binary  images  are  typically  unsuitable  to  be  a  cover 
image  because  of  their  extremely  low  color  depth  and  their 
intolerance  to  noise  adding. 

Intensity  Images 

Each  pixel  is  represented  by  an  intensity  value  between  0  (black) 
and  255  (white).  Intensity  images  offer  moderate  capacity  for 
embedded  data.  For  example,  if  a  least  significant  bit  encoding 
scheme  was  used,  one  can  embed  (M  x  A^/8)  8-bit  data  symbols  into 
an  NxM  intensity  image.  Increase  a  pixel  value  by  one,  in  a  least 
significant  bit  encoding  scheme,  introduces  a  0.39%  change  in  its 
intensity.  This  change  is  visually  imperceptible. 

RGB  Images 

Similar  to  intensity  images.  A  pixel  can  be  represented  in  two 
different  ways:  (1)  as  a  tuple  of  three  intensities  (red,  green,  and 
blue),  or  (2)  as  three  separate  color  planes.  RGB  images  offers  as 
much  as  three  times  the  capacity  of  similar  intensity  images. 

Indexed  Images 

Consists  of  two  arrays,  an  image  matrix  and  a  colormap  (or  palette). 
The  colormap  is  an  ordered  set  of  values  that  represents  the  colors  in 
the  image.  For  each  image  pixel,  the  image  matrix  contains  a  value 
that  is  an  index  into  the  colormap.  The  most  common  of  this  type  of 
image  are  CompuServe’s  Graphics  Interchange  Format  (GIF) 
images.  Data  embedding  techniques  for  indexed  images  embed  data 
either  in  the  image’s  palette  or  in  the  image’s  data. 

Data  embedding  in  digital  images  is  possible  because  the  gaps  in  human  visual  systems  can  be 
easily  exploited.  In  the  case  of  images,  the  human  eye  is  relatively  insensitive  to  high 
frequencies.  This  fact  has  been  utilized  in  many  data  embedding  algorithms.  For  example,  one 
can  modify  the  least  significant  bit  of  the  gray  levels  in  a  digital  image  to  embed  a  message. 
Least  significant  bit  encoding  is  a  high  frequency  operation  that  introduces  as  little  as  a  Xse 
change  to  roughly  50%  of  an  image’s  pixels.  The  human  eye  is  also  relatively  insensitive  to 
gradual  changes  in  shades.  Because  of  this,  bits  of  information  can  be  embedded  into 
coefficients  of  image  transforms,  such  as  the  Discrete  Cosine  Transform  or  the  Fourier 
Transform,  which  represent  can  signals  as  a  weighted  summation  of  sine  and  cosine  functions. 
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Studies  also  show  that  the  human  eye  is  less  sensitive  to  changes  in  the  blue  channel  in  an  RGB 
image  and  to  the  overall  luminance  channel  [Hons98].  These  properties  of  the  human  visual 
system  can  be  exploited  to  hide  information  into  a  digital  image.  It  must  be  pointed  out, 
however,  that  exploiting  the  gaps  of  the  human  visual  systems  can  only  be  successful  if  the 
ultimate  consumer  of  the  image  is  the  human  eye.  Imperceptibility,  which  is  related  to  the 
human  detector,  is  not  equivalent  to  undetectability,  which  belongs  to  the  field  of  statistics. 
Simply  because  a  human  cannot  visually  perceive  an  embedded  message  does  not  necessarily 
imply  that  a  computer  cannot  find  a  statistical  anomaly  introduced  by  the  embedded  information 
in  the  image. 

Although  the  general  data  embedding  scenario  is  simple,  assumptions  regarding  the 
communication  channel,  implementation  issues,  and  data  embedding  requirements  must  be 
formalized  and  addressed  before  any  discussion  about  data  embedding  techniques  can  be 
meaningful.  As  one  will  see,  the  relationship  between  the  assumptions  made,  the  requirements 
imposed,  and  the  design  implemented  interact  with  one  another.  Therefore,  tradeoffs  are  always 
weighed. 

2.2  Assumptions  on  the  Communication  Channel 

One  assumption  is  the  stability  of  the  communication  channel.  Is  Willie  a  passive  or  active 
warden?  A  passive  warden  merely  attempts  to  discover  the  presence  (perhaps  in  a  probabilistic 
setting)  of  embedded  data  in  the  cover  object.  An  active  warden  tries  to  reduce  the  channel 
capacity  of  the  covert  channels  in  order  to  deny  any  hidden  message  from  reaching  its  intended 
recipient.  The  assumption  made  on  the  stability  of  the  communication  channel  directly  affects 
the  robustness  requirement  (a.k.a.  data  persistence)  on  the  data  embedding  technique.  If  a 
passive  warden  is  assumed,  robustness  is  not  an  issue  ~  it  is  assumed  that  the  stego  object  will 
not  be  subject  to  distortion  or  removal  attacks.  If  an  active  warden  is  assumed,  it  is  assumed  that 
the  stego  object  will  undergo  some  types  of  distortion  and  that  the  plain  object  must  still  be 
recoverable  by  the  recipient. 
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Another  is  the  assumption  of  the  capacity  of  the  communication  channel.  How  large  are  the 
private  messages  with  respect  to  the  images  into  which  these  message  are  embedded?  In 
determining  the  length  of  the  message,  one  must  differentiate  between  the  information  bits  (the 
logical  bits  that  comprise  the  message)  and  the  data  bits  (the  physical  bits  that  encode  the 
message).  A  short  message  (small  number  of  information  bits)  may  be  encoded  redundantly 
throughout  the  image  or  may  be  encoded  using  an  error  correction  code  (large  number  of  data 
bits).  The  higher  the  number  of  physical  data  bits  that  comprise  the  message,  the  greater 
chances  the  message  introduces  detectable  artifacts  into  the  cover  object.  As  such,  capacity 
affects  imperceptibility  and  detectability  requirements.  Imperceptibility  deals  with  visual 
artifacts  that  may  be  introduced  into  the  cover  object.  Detectability  deals  with  statistical  analysis 
of  the  stego  object  in  an  attempt  to  deduce  whether  or  not  the  object  contains  an  embedded 
message. 

A  third  assumption  is  the  authentication  of  the  communicating  parties.  What  is  the  possibility 
that  the  (active)  warden  will  attempt  to  spoof  one  of  the  communicators?  This  assumption  lays 
down  the  requirements  for  security  and  invertibility.  Security  is  normally  based  on  resistance  to 
attacks  to  obtain  the  secret  key,  as  it  is  in  a  cryptographic  framework.  However,  it  also  includes 
attacks  that  use  the  knowledge  of  the  embedding  and  extracting  algorithm  to  obtain,  detect, 
disrupt,  or  remove  the  embedded  message. 

2.3  Data  Embedding  Implementation  Issues 

Data  embedding  schemes  can  be  categorized  on  how  they  address  certain  data  embedding 
implementation  issues.  These  issues  are; 

1 .  whether  the  embedded  data  is  privately  or  publicly  extractable, 

2.  whether  the  original  image  is  required  to  extract  the  embedded  data,  and 

3.  whether  the  data  can  be  extracted  asymmetrically  or  not. 


11 


Private  or  Public  Marking 

Private  marking  systems  are  characterized  in  the  following  manner:  one  can  only  decide  whether 
certain  data  is  embedded  or  not.  Private  marking  schemes,  popular  among  watermarking 
schemes,  typically  utilize  a  simple  hypothesis  test: 

hypothesis  Hq'.  the  watermark  w  is  not  present 
hypothesis  H\ :  the  watermark  w  is  not  present 

The  problem  of  hypothesis  testing  is  to  decide  which  of  the  hypothesis  is  true,  when  a  stego 
object  Y is  given.  Usually  it  is  not  possible  to  separate  all  watermarked  and  unwatermarked 
objects  perfectly:  a  received  object  Y might  be  watermarked  with  probability piHi\Y)  or  not 
watermarked  with  probability  p(Ho\ Y).  We  trade  off  the  probability  pfp  of  accepting  Hi  when  Ho 
is  true  (false  positive)  and  the  probability  pf^  of  accepting  Hq  when  it  is  false  (false  negative). 
Bayes’  solution  is  the  decision  rule 

Py(Y\H^)  1>K=^  accept  H, 
p^(Y\H^)\<K=^  accept  H^ 

where  K  =  cos\.pFppHol{cosipFNpHi)  is  a  constant  depending  on  the  a  priori  probabilities  for  H\ 
and  Ho  and  the  cost  connected  with  the  different  decision  errors  [Egge99]. 

In  the  copyright  protection  architecture,  for  example,  given  an  ownership  mark  m  and  an  image  7, 
one  can  either  detect  or  not  detect  the  presence  of  m  in  7.  Note  that  with  a  private  marking 
scheme,  there  is  no  means  to  detect  the  ownership  code  if  some  hypothesis  about  the  possible 
owner  cannot  be  done.  It  is  only  possible  to  verify  if  the  image  belongs  to  a  particular  author 
whose  ownership  code  is  known.  An  example  of  a  private  marking  scheme  is  presented  in 
[Cox95a].  A  watermark  is  created  using  a  private  key  and  is  embedded  into  an  image.  The 
detector  recreates  the  watermark  using  the  same  private  key  and  attempts  to  detect  it  within  a 
possibly  attacked  version  of  the  image. 
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A  weakness  of  the  detectable  watermarking  scheme  is  presented  in  [Adel99].  The  result  of  an 
ownership  dispute  is  the  decision  made  by  an  arbitrator  after  comparing  several  claims  of 
ownership.  This  result  does  not  determine  the  rightful  ownership  in  general,  since  the  rightful 
copyright  holder  might  not  be  participating  in  the  corresponding  dispute.  Take  two  principals, 
Alice  and  Bob,  who  both  claim  image  /  belongs  to  them.  In  legal  arbitrary,  Alice  and  Bob  will 
present  their  watermarks,  wa  and  wb,  and  Judge  Judy  will  determine  which  watermark  can  be 
detected  in  I.  If  the  image,  however,  really  belongs  to  a  third  principal  Mark,  Judy  will  not  be 
able  to  determine  that  Mark’s  watermark  is  in  the  image. 

Techniques  that  allow  the  embedded  data  to  be  read  directly  from  the  image  is  referred  to  as 
being  a  public  marking  scheme.  Without  prior  knowledge  of  the  embedded  mark  or  the  original 
image,  public  marking  systems  extract «  bits  of  information  from  the  marked  image.  In  a 
copyright  protection  architecture,  the  ownership  information  can  be  read  by  anyone  without  prior 
knowledge  of  whose  watermark  to  look  for.  Information  is  embedded  and  subsequently  retrieved 
with  a  (publicly  available)  key. 

Public  marking  schemes  are  characterized  by  embedding  multiple  marks  that  correspond  to 
symbols  in  an  alphabet.  These  marks  may  correspond  to  digital  pointers,  URLs,  or  ownership 
codes.  [Piva97]  and  [Ruan98]  present  a  public  marking  technique. 

Watermark  Blindness 

Particular  attention  must  be  paid  attention  to  the  algorithm  used  to  recover  the  embedded  data 
from  the  image.  In  some  cases,  the  embedded  data  is  extracted  by  comparing  the  marked  version 
of  the  object  to  the  non-marked  one  [Cox95a,  Swan96b,  Wolf96].  With  the  original  object,  one 
can  opt  to  preprocess  the  object  under  test  to  compensate  for  any  distortion.  Although  these 
techniques  may  be  very  robust  with  respect  to  geometric  distortions,  the  availability  of  the 
original  multimedia  object  may  not  be  possible. 
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Techniques  that  extract  the  embedded  data  without  resorting  to  the  comparison  between  marked 
and  non-marked  objects  are  said  to  be  oblivious  or  blind.  Several  blind  image  watermarking 
algorithms  have  been  proposed  and  implemented,  but  none  of  them  is  able  to  survive  geometric 
manipulations.  Current  research  efforts  have  suggested  embedding  a  reference  mark  in  addition 
to  the  watermark  itself  [Bend96,  Hons98,  Pere99].  The  reference  mark  is  to  be  used  for 
identifying  the  type  of  geometric  distortion  the  object  undergone.  The  resistance  to  geometric 
manipulations  (e.g.,  translation,  resizing,  rotation,  cropping,  and  warping)  is  still  an  open  issue 
and  a  solution  must  be  found  before  watermarking  techniques  are  successfully  applied  to  image 
copyright  protection.  [John99b]  proposed  a  method  for  the  recovery  of  original  size  and 
appearance  of  images  based  on  the  concept  of  identification  marks.  This  method  does  not 
require  the  original  image,  but  only  a  small  number  of  salient  image  points.  With  these  salient 
image  points,  it  is  possible  to  undo  the  affine  transformation  an  image  had  xmdergone.  However, 
it  is  contested  that  the  salient  image  points  required  are  equivalent  to  having  the  original  image 
and  therefore,  any  watermarking  scheme  based  on  this  method  is  not  truly  oblivious.  Self¬ 
synchronizing  digital  watermarks  are  also  being  explored  with  hopes  of  being  resistant  to 
geometric  distortions  without  requiring  the  original  image  or  a  database  of  salient  image  points 
[Algh99a,  99b,  99c]. 

Symmetric  or  Asymmetric  Extraction 

A  technique  is  said  to  be  symmetric  if  the  stego-key  used  in  the  embedding  process  is  the  stego- 
key  used  for  extraction.  Asymmetric  techniques  use  different  keys.  It  is  widely  believed  that 
asymmetric  mechanisms  are  likely  to  be  significantly  more  robust  than  symmetric  ones.  Once 
the  embedded  data  has  been  read  with  the  symmetric  key,  it  will  be  much  easier  for  an  attacker 
to  remove  it  or  to  make  it  unreadable.  For  example,  one  can  encode  the  inverse  of  the  embedded 
data. 

[Crav99]  contends  that  the  keys  used  in  all  watermarking  schemes  eventually  become  public 
knowledge.  Therefore,  a  pure  asymmetric  extraction  technique  does  not  exist.  During  an 
ownership  dispute,  the  two  parties  involved  in  the  arbitration  must  produce  their  extraction  key 
in  order  to  determine  whose  watermark  is  contained  in  the  object.  Their  keys  now  become 
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public  and  they  may  be  used  to  remove  ownership  marks  in  other  previously  marked  objeets.  To 
avoid  this  eventual  publieation  of  secret  keys,  zero-knowledge  deteetion  sehemes  are  currently 
being  investigated  to  provide  a  totally  private  scheme  [Crav99]. 

2.4  Requirements  on  Data  Embedding  Schemes 

As  alluded  to  in  the  discussion  on  the  assumptions  made  on  the  communication  channel, 
requirements  on  data  embedding  techniques  are  applieation-dependent  and,  for  every  applieation, 
these  requirements  vary  in  degree  and  have  to  be  balanced  against  other  competing  requirements. 
As  listed  in  [Piva98],  these  requirements  are: 

1 .  security, 

2.  imperceptibility, 

3.  detectability  with  low  error  probability, 

4.  payload, 

5.  robustness,  and 

6.  invertibility. 

In  evaluating  data  embedding  schemes,  one  must  consider  the  requirement  tradeoffs  that  the 
developers  may  have  made.  For  example,  the  ability  to  discern  images  with  embedded  messages 
is  directly  influenced  by  the  physical  length  of  the  message,  as  well  as  the  format  and  content 
and  size  of  the  carrier  image.  Obviously,  the  longer  the  message  (high  capacity),  the  larger  the 
modification  to  the  earner  image  and  the  higher  the  probability  that  the  modifications  can  be 
statistically  detected  (low  imperceptibility).  Beeause  of  this  tradeoff,  one-bit  watermarking 
schemes  have  a  distinct  advantage  over  an  n-bit  watermarking  scheme.  The  ehoice  of  the  carrier 
image  is  also  crucial.  Natural  photographs  with  24  bits  per  pixel  provide  the  best  environment 
for  message  hiding,  especially  if  they  contain  highly  textured  areas.  The  redundancy  of  the  data 
in  these  areas  helps  to  conceal  the  presence  of  the  embedded  messages.  Lossy  compressed 
images,  such  as  JPEG  files,  are  more  sensitive  to  small  perturbations  of  the  image  data,  and  pose 
a  challenge  for  creating  a  seeure  data  embedding  technique  with  reasonable  capacity.  Indexed 
images,  although  abundant  over  the  Internet,  also  provide  a  hostile  environment  for  the 
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steganographer.  The  limited  number  of  available  colors  imposed  by  the  finite  palette  makes  the 
process  of  message  hiding  a  difficult  challenge. 


Imperceptibility  and  robustness  also  compete  with  each  other.  Tlie  performance  of  the 
watermark  detector,  for  example,  generally  increases  with  the  energy  of  the  watermarks  inserted: 
the  stronger  the  watermark,  the  better  the  watermark  detector  performs.  However,  the  stronger 
the  watermark,  the  more  likely  it  is  to  produce  visible  artifacts^.  In  addition,  if  the  watermarking 
scheme  is  overly  conservative  to  guarantee  transparency  over  a  variety  of  input  images,  it  is 
likely  that  the  watermarks  will  not  be  detected  after  some  distortion  operation  or  attack  is  applied 
to  it. 

Security 

The  security  of  a  data  embedding  scheme  cannot  be  based  on  algorithm  ignorance.  As  for 
cryptography  [Kerkl883],  it  is  well  known  that  the  effectiveness  of  an  algorithm  can  not  be 
based  on  the  assumption  that  possible  attackers  do  not  know  how  the  data  has  been  embedded 
into  the  multimedia  object.  Nevertheless,  the  robustness  of  most  commercial  digital 
watermarking  products  is  based  on  such  an  assumption.  Though  some  of  them  claim  to  be 
exceptionally  resistant,  by  knowing  how  the  watermark  encoder  and  decoder  work,  it  is  very 
easy  for  a  skilled  hacker  to  make  the  watermark  unreadable  or  imdetectable  [Peti98]. 

In  covert  communication  applications,  undetectability  of  the  embedded  message  also  falls  under 
the  security  aspect.  Even  with  the  knowledge  of  the  encoder  and  decoder,  one  should  not  be  able 
to  detect  the  artifacts  introduced  during  message  embedding.  We  do  not  require  that  the  hidden 
message  is  recovered  for  a  steganalytic  attack  to  be  successful  -  the  hidden  message  must  only 
be  detected  for  the  warden  to  defeat  the  steganographer^. 


^  Perceptual  masking  has  made  it  possible  to  increase  watermarking  strength  without  increasing 
the  visibility  of  artifacts,  but  the  problem  remains. 

®  The  question  of  then  decrypting  the  recovered  message  is  a  classical  cryptanalytic  question  and 
lies  outside  the  interests  of  steganalysis  proper. 
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[Fran99]  discusses  possible  attacks  in  order  to  evaluate  the  security  of  a  data  embedding  system. 
The  paper  lists  seven  types  of  attacks,  categorized  into  passive  and  active  attacks.  Covert 
communication  applications  mainly  consider  passive  attacks,  as  pointed  out  in  [Ande98].  The 
discussion  of  active  attacks  is  common  for  watermarking  systems  [Petit98]. 

Passive  Attacks 

A  passive  attacker  (passive  warden  scenario)  is  restricted  to  the  statistical  analysis  of  the  stego 
object  in  order  to  detect  the  presence  of  an  embedded  message.  The  nature  and  amount  of  data 
the  attacker  has  access  to  define  the  type  of  attack  (Table  2).  [John98,  98b,  98c,West99]  provides 
security  analysis  and  reveals  the  security  flaws  of  available  data  embedding  software. 


Table  2.  Description  of  Passive  Attacks  on  Data  embedding  Systems. 


Type  of  Passive  Attack  Description  _ 

Stego-Only  Attack  Attacker  analyzes  the  intercepted  stego  object  X 

Stego*  Attack  The  user  has  repeated  embedded  in  the  same 

cover  object;  attacker  has  access  to  all  resulting 
stego  objects  Xi,  X:, . . .,  Xn 

Cover-Stego  Attack  In  addition  to  the  intercepted  stego  X,  the  attacker 

has  access  to  the  cover  object  U 

Emb-Stego  Attack  The  attacker  knows  both  the  plain  object 

(message)  V  and  the  resulting  stego  object  X 


Cover-Emb-Stego  Attack  The  attacker  knows  the  cover  object  U,  the  plain 

object  V,  and  the  resulting  stego  object  X 


Active  Attacks 

An  active  attacker  (active  warden  scenario)  is  allowed  to  modify  the  cover  or  stego  object  in 
order  to  disrupt  the  data  embedding  communication  channel  (Table  3).  Disruption  of  the  channel 
is  achieved  if  the  attacker  renders  the  embedded  message  unextractable. 
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Table  3.  Description  of  Active  Attacks  on  Data  embedding  Systems. 


Type  of  Active  Attack 
Manipulation  of  Stego 
Object 


Manipulation  of  Cover 
Object 


Description  _ _ _ 

The  stego  object  can  be  manipulated  to  prevent 
the  transmission  of  the  embedded  message.  On 
one  hand,  this  attack  attempts  to  foil  the  secret 
communication.  On  the  other  hand,  the  attacker 
can  analyze  the  reaction  of  the  attacked  parties:  if 
they  try  to  send  another  possible  stego  object,  it 
could  be  a  sign  that  data  embedding  is  being  used. 

The  attacker  can  try  to  make  his  attack  easier  by 
introducing  potential  cover  objects  into  the  user  s 
database  of  cover  objects.  This  attack  is  similar 
to  the  Cover-Stego  attack,  but  the  attacker  takes 
an  active  role  in  acquiring  the  cover  object. 


Imperceptibility^ 

Information  embedded  into  digital  imagery  must  not  introduce  visible  artifacts  in  the  image. 

This  is  extremely  important  for  professional  photographers  and  movie  producers,  especially  if 
the  data  embedding  technique  is  to  be  used  in  a  copyright  protection  architecture.  Howeyer,  the 
attempt  to  render,  for  example,  digital  watermarks  inyisible  to  the  human  obseryer  conflicts  with 
other  requirements.  In  current  watermarking  techniques,  for  example,  the  strength  of  the 
watermark  is  directly  related  to  its  yisibility.  The  greater  the  strength  of  the  watermark,  the 
greater  its  visibility  and  the  better  the  watermark  detector  performs.  The  use  of  perceptual  masks 
has  allowed  stronger  watermarks  to  be  embedded  into  an  image  without  substantially  degrading 
the  image.  In  a  covert  communication  scenario,  imperceptibility  (and  undetectability)  would  be 
paramount  even  at  the  expense  of  capacity. 


Most  evaluations  of  proposed  data  embedding  schemes  are  inadequate  in  gauging  the 
perceptibility  of  the  produced  visual  artifacts.  Typically  they  rely  on  a  person  (or  a  group  of 
people)  yoting  on  whether  or  not  a  deterioration  in  image  quality  can  be  visually  perceived. 
What  is  desired  in  this  area  is  an  adequate  computer  model  of  the  human  visual  system  that  can 
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be  integrated  into  a  data  embedding  scheme  such  that  the  creation  of  visual  artifacts  can  be 
minimized  or  avoided  without  human  interaction.  Two  human  visual  models  are  available  in  the 
research  arena:  one  proposed  by  [Giro89];  one  used  by  [Podi98]. 

Low  Error  Probability 

Even  in  the  absence  of  attacks  or  signal  distortions,  the  probability  of  failing  to  detect  the 
embedded  data  (false  negative)  and  of  extracting  data  when  it  does  not  exist  (false  positive)  must 
be  very  small.  It  is  unacceptable  in  a  legally  binding  arbitration  for  a  watermarking  scheme,  for 
example,  to  have  a  high  probability  of  error.  In  a  watermarking  scenario,  an  acceptable 
probability  of  false  positives  is  1 0'^. 

Payload 

When  speaking  of  how  many  bits  are  embedded  in  an  image,  one  must  draw  a  distinction 
between  data  bits  and  information  bits*.  For  example,  it  is  sufficient  to  embed  one  bit  of 
information  for  digital  watermarking  schemes.  In  a  private  watermarking  scheme,  this  one  bit  of 
information  could  be  the  answer  to  "Does  Mr.  X  own  this  digital  image?"  However,  most  digital 
watermarking  schemes  encode  this  one-bit  of  information  redundantly  throughout  the  image  to 
combat  the  effects  of  possible  image  processing  operations  on  the  image.  Therefore,  the  actual 
amount  of  data  embedded  is  greater  than  one  bit. 

With  this  in  mind,  special  care  must  be  taken  in  comparing  data  embedding  techniques.  One 
cannot  compare  the  robustness  aspect  of  a  data  embedding  technique  that  merely  hides  one  bit  of 
data  against  another  data  embedding  technique  that  hides  one  bit  of  information  (possible 
encoded  n  times  in  the  image  data).  [Frid99d]  proposes  a  scheme  to  compare  detectable,  one-bit 
watermarking  techniques  with  readable,  n-bit  watermarking  techniques. 


^  Imperceptibility,  which  relates  to  the  human  visual  system,  is  different  fi'om  undetectability, 
which  relates  to  the  statistical  analysis. 

*  The  terms  "information"  and  "data"  are  not  interchangeable  in  this  discussion. 
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In  spite  of  the  importance  of  knowing  how  many  bits  can  be  embedded  into  a  digital  image  of  a 
given  size,  such  an  issue  has  not  been  satisfactory  addressed  yet.  In  fact,  an  upper  bound  to  the 
amount  of  information  that  can  be  reliably  embedded  into  an  image  of  a  given  dimension  is  not 
known.  A  number  of  papers  attempt  to  address  this  issue  [Smit96,  Serv98]. 

Robustness 

Digital  multimedia  objects  may  undergo  numerous  types  of  distortions  during  their  lifecycle. 

For  digital  imagery,  these  distortions  include  lossy  compression,  filtering,  resizing,  contrast  and 
brightness  enhancement,  cropping,  and  rotation.  Depending  on  the  assumptions  made 
concerning  the  type  of  warden  in  control  of  the  communication  channel,  embedded  data  may  be 
required  to  be  persistent  even  after  its  cover  object  had  undergone  distortions.  In  the  case  of 
digital  watermarking  for  copyright  protection,  the  ownership  mark  should  be  detectable  even 
after  these  distortions  occurred. 

Resistance  to  geometric  manipulations  is  very  important  because  they  do  not  severely  degrade 
the  quality  of  the  image.  They  can  be  applied  to  make  the  watermark  unreadable.  For  example, 
if  an  image  is  cropped  by  one  column  or  by  one  row  or  is  rotated  0.5  degrees,  or  scaled  to  101 
the  difference  to  the  original  image  is  irrelevant  and  imperceptible.  However,  the  embedded  data 
may  no  longer  be  detected  or  read.  Recent  efforts  have  attempted  to  embed  a  regular  pattern  into 
the  image  in  order  to  assist  in  the  detection  of  these  geometric  distortions  [Bend96,  aHons98, 
Adel99].  As  mentioned  earlier,  [John99b]  proposed  a  method  for  the  recovery  of  original  size 
and  appearance  of  images  based  on  the  concept  of  identification  marks.  However,  it  is  contested 
that  the  availability  of  the  salient  image  points  is  equivalent  to  having  the  original  image.  Other 
efforts  have  been  directed  in  creating  self-synchronizing  watermarks  [Algh99a,  99b,  99c]. 
Resistance  to  geometric  manipulations  (e.g.,  translation,  resizing,  rotation,  cropping,  and 
warping)  is  still  an  open  issue  and  a  solution  must  be  found  before  watermarking  techniques  are 
successfully  applied  to  image  copyright  protection  [Ruan97]. 

One  might  attempt  to  increase  the  robustness  of  a  data  embedding  system  by  trying  to  foresee  all 
possible  attacks.  However,  [Peti99]  points  out  that  given  a  data  embedding  system,  one  can 
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invent  a  distortion  (or  a  combination  of  distortions)  that  will  prevent  the  recovery  of  the 
embedded  data  while  leaving  the  perceptual  value  of  the  stego  object  undiminished.  To  prove 
this  point  and  for  research  purposes,  Fabien  Petitcolas  developed  the  Stirmark  attack  which 
applies  a  variety  of  distortions  to  an  image  without  degrading  its  quality  [Peti98b].  Examples  of 
what  Stirmark  does  are  given  in  Figure  5. 


After  Stirmark  Original  Image  B 


After  Stirmark 


Original  Image  A 


Figure  5.  Results  of  a  Stirmark  Attack. 

[Peti99]  offers  a  good  explanation  of  Stirmark: 

Stirmark  applies  minor  unnoticeable  geometric  distortions.  The  image  is  slightly 
stretched,  sheared,  shifted,  bend,  and  rotated  by  unnoticeable  random  amounts.  A  slight 
random  low  frequency  deviation,  which  is  greatest  at  the  center  of  the  image,  is  applied 
to  each  pixel.  A  higher  frequency  displacement  of  the  form 
A,  sin(w^x)  sin(w^y)  +  n{x,  y)  —  where  n(x,y)  is  a  random  number  —  is  also  added. 

Finally,  a  transfer  function  that  introduces  a  small  and  smoothly  distributed  error  into  all 
sample  values  is  applied.  This  emulates  the  small  non-linear  analog-to-digital  converter 
imperfections  typically  found  in  scanners  and  display  devices. 


For  more  examples  how  Stirmark  distorts  an  image,  refer  to 

http://www.cl.cam.ac.uk''users/fapp2/steganographv/image  watenTiarking/stin'nark/samples2.html. 

Invertibility 

The  definition  for  watermark  invertibility  is  given  by  [Crav97].  In  their  paper.  Graver  et  al 
analyzes  the  possibility  of  invalidating  ownership  claims  supported  by  watermarking  by  reverse 
engineering  the  watermarking  process.  We  say  that  a  watermark  is  invertible  if: 
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1 .  It  is  possible  to  generate  a  false  watermark  and  a  fake  original  document  which  is 
perceptually  equal  to  the  true  one,  and 

2.  The  false  watermark  and  the  true  watermark  can  be  found  in  both  the  fake  original 
and  in  the  true  original. 

Take,  for  example,  the  following  an  additive  watermarking  scheme.  Given  an  image  /,  Alice 
embeds  her  watermark,  wa,  by 

/'  =  /  +  w. 

Then  Bob  can  create  a  fake  original  document  /"by  constructing  a  false  watermark,  wb,  and 
subtracting  it  /’. 

/It  jf 

=  7 -w^ 

/’’is  Bob’s  forged  original  image,  /'contains  both  watermarks,  wa  and  wb,  depending  on  which 
original  image  is  used  to  claim  ownership. 

r  =  I  +  w^  =1''+Wg 

In  addition,  Alice’s  original  image  contains  Bob’s  watermark  and  Bob’s  forged  original  contains 
Alice’s  watermark. 

r  =  I  +  w^-w, 

In  a  legal  arbitration,  a  deadlock  ensues  since  it  is  not  possible  to  determine  which  is  the  real 
original  image  and  which  is  the  fake  one. 

For  digital  watermarking  for  copyright  protection  and  for  source  authentication  applications,  the 
watermark  should  not  be  invertible.  Current  watermarking  techniques  use  watermarks  that  are 
dependent  on  the  image  (by  using  the  hash  of  the  image,  for  example)  and  on  a  secret  key.  This 
precludes  the  creation  of  a  false  watermark  that  can  be  foimd  in  the  true  original  by  making  such 
creation  computationally  infeasible. 
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3  Sample  Data  Embedding  Techniques 

This  section  provides  examples  of  spatial  and  spectral  data  embedding  techniques.  The  spatial 
techniques  were  mainly  developed  for  covert  communication  applications  where  capacity  was 
maximized  and  a  passive  warden  scenario  was  assumed.  The  two  spectral  examples  were 
developed  for  digital  watermarking  applications  wherein  the  embedded  watermarks  were 
required  to  be  robust  with  respect  to  active  attacks.  Note  that  data  embedding  techniques  for  one 
type  of  image  may  not  be  applicable  to  another  type  of  image.  On  one  hand,  data  embedding 
techniques  for  indexed  images  cannot  be  used  for  intensity  images.  On  the  other  hand, 
techniques  for  intensity  images  may  be  modified  to  work  on  RGB  images.  Binary  images  are 
typically  unusable  as  cover  objects  due  to  their  extremely  low  color  depth  and  their  intolerance 
to  noise  adding.  They  still  may  be  used,  however,  if  an  embedding  technique  is  developed  to  use 
slight  geometric  distortions  to  encode  a  message. 

3.1  Spatial  Techniques 

Data  can  be  embedded  in  both  the  spatial  domain  and  the  frequency  domain.  Spatial  domain 
techniques  are  easy  to  implement.  However,  it  often  fails  under  signal  processing  attacks  such  as 
filtering  and  compression.  Spatial  techniques  generally  have  higher  payloads  than  spectral 
techniques,  but  are  less  robust  with  respect  to  most  of  the  signal  processing  attacks. 

Additionally,  frequency  domain  techniques  tend  to  have  better  performance  than  spatial  domain 
techniques  since  most  perceptual  models  are  developed  in  the  frequency  domain  [Podi98]. 

LSB  Encoding  in  Intensity  Images 

Natural  photographs  with  24  bits  per  pixel  provide  the  best  environment  for  message  embedding. 
The  redundancy  of  the  data  helps  conceal  the  presence  of  the  data  [Frid99e].  A  simple  method 
of  a  spatial  data  embedding  technique  for  8-  or  24-bit  intensity  images  uses  least  significant  bit 
(LSB)  encoding. 
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Using  intensity  images  as  carriers,  one  can  replace  iV  least  significant  bitplanes  of  an  image  with 
A'^bitplanes  can  contain  a  message^.  For  example,  one  can  replace  the  least  significant  bits  (N- 
1)  of  a  grayscale  intensity  image  with  the  bits  of  a  message.  At  most,  this  would  change  a  pixel’s 
intensity  level  by  1/256  or  0.39%.  With  N=3,  one  replaces  the  least  three  significant  bitplanes 
with  the  bits  of  a  message.  This  would  change  a  pixel’s  intensity  level  by  at  most  3/256  (1.17%). 

Figure  6  illustrates  how  the  replacement  of  least  significant  bitplanes  of  a  grayscale  intensity 
image  affects  the  quality  of  the  image.  In  each  case.  A/ least  significant  bitplanes  were  replaced 
with  a  random  bit  pattern  of  zeros  and  ones.  Even  atN=  3,  which  introduces  at  most  1.17% 
change  in  pixel  intensity,  one  can  see  visual  artifacts  (especially  in  areas  of  smooth,  gradual 
changes)  in  the  resulting  images.  As  in  any  data  embedding  technique,  one  must  perform  a 
trade-off  between  capacity  and  invisibility.  One  can  embed  three  times  as  many  bits  with  N=  3 
than  if  N  was  restricted  to  A  =  1 . 

One  can  extend  this  simple  LSB  encoding  by  embedding  message  bits  into  groups  of  pixels.  For 
example,  one  can  use  odd-even  parity  to  embed  the  message  bit  into  a  group  of  pixels.  Let  the 
parity  of  a  group  of  pixels  be  defined  as  the  sum  of  the  individual  pixel’s  parity  modulo  2.  If  the 
message  bit  is  1,  one  forces  the  parity  of  the  group  of  pixels  to  be  odd,  for  example.  If  the 
message  bit  is  0,  then  one  coerces  the  parity  of  the  group  to  be  even.  Because  one  has  the 
fi’eedom  to  choose  which  pixel  to  modify  in  order  to  change  the  group  parity,  one  can  select  the 
pixel  that  will  introduce  the  least  amount  of  distortion. 


^  Without  the  loss  of  generality,  a  "message"  is  considered  a  simple  bitstream  of  data.  This 
bitstream  could  be  a  (encrypted)  message,  another  image,  or  a  binary  file. 
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Figure  6.  Effects  of  LSB  Encoding. 

Data  embedding  techniques  that  use  LSB  encoding  are  easy  to  detect  [Aura96]  and  trivial  to 
remove  [Ande98].  Almost  any  trivial  filtering  operation  will  affect  the  value  of  many  of  the 
least  significant  bits,  effectively  destroying  any  message  encoded  in  the  least  significant  bits.  To 
enhance  LSB  encoding  robustness  properties,  one  possible  technique  is  to  use  redundancy  — 
either  to  apply  an  error  correction  code  or  simple  embed  the  message  a  large  number  of  times. 
However,  an  easy  way  to  attack  LSB  encoding  based  on  this  countermeasure  is  to  break  up  the 
synchronization  needed  to  locate  the  samples  in  which  the  message  is  hidden  [Ande98].  Digital 
images,  for  example,  can  be  cropped,  scaled,  or  rotated. 

In  some  applications,  however,  the  LSB  encoding  is  extreme  sensitivity  to  modification  may  be  a 
blessing  rather  than  a  curse. 

Steve  Walton  proposed  an  image  authentication  algorithm  based  on  the  fragility  of  messages 
embedded  in  digital  images  using  LSB  encoding.  In  [Walt95],  he  proposes  a  technique  that  uses 
a  key-dependent  pseudo-random  walk  on  the  image.  The  checksum  is  obtained  by  summing  the 
numbers  determined  by  the  seven  most  significant  bits  of  each  pixel  along  the  random  walk  and 
taking  remainder  operation  with  a  large  integer  N.  The  checksum  is  inserted  in  a  binary  form  in 
the  LSB  of  selected  pixels.  The  checksum  is  constructed  from  the  seven  most  significant  bits 
because  the  LSBs  of  the  pixels  used  to  embed  the  checksum  cannot  be  allowed  to  contribute  to 
the  checksum  calculation. 

To  check  the  authenticity  of  an  image,  one  essentially  reverses  the  embedding  process:  one 
extracts  the  embedded  checksum  from  the  selected  pixels  and  calculates  the  checksum  based  on 


the  seven  most  significant  bits  along  the  re-generated  random  walk.  If  the  recovered  checksum 
matches  the  calculated  checksum,  the  image  has  not  been  tampered  with. 

[Walt95]  suggests  that  this  could  be  repeated  for  many  disjoint  random  walks  or  for  one  random 
walk  that  goes  through  all  pixels.  To  prevent  tampering  based  on  exchanging  groups  of  pixels 
with  the  same  checksum,  the  checksum  can  be  made  “walk-dependent”.  The  method  is  very  fast 
and  on  average  modifies  only  half  of  the  pixels  by  one  gray  level.  [Bald99]  extended  this 
algorithm  by  subdividing  the  entire  image  into  sub-blocks  and  ultimately  protecting  each  sub¬ 
block  by  introducing  the  walk-dependent  checksum.  [Bald99]  pointed  out,  however,  that  this 
extension  could  be  defeated  by  simply  replacing  a  block  with  its  corresponding  block  from 
another  image. 

Although  checksums  can  provide  a  very  high  probability  of  tamper  detection,  they  cannot 
distinguish  between  a  malicious  attack  (e.g.,  feature  replacement  or  modification)  and  an 
innocent  manipulation  of  the  image  (e.g.,  lossy  compression,  and  brightness  or  contrast 
adjustment).  Increasing  the  gray  scales  of  all  pixels  by  one  would  indicate  a  large  extent  of 
tampering,  although  the  image  content  has  been  unchanged  for  all  practical  purposes. 

Data  embedding  Techniques  for  Indexed  Images 

Messages  can  be  embedded  into  indexed  images  in  either  the  image  palette  or  the  image  data. 
Gifshuffle  [Kwan]  embeds  messages  into  the  palette  by  permuting  its  entries.  This  method  does 
not  change  the  appearance  of  the  image,  which  is  certainly  an  advantage,  but  its  security  is  weak 
because  palettes  are  normally  ordered  by  frequency  of  occurrence  or  luminance  or  some  other 
scalar  factor.  A  randomly  ordered  palette  is  suspicious  and  easily  detectable. 

Other  available  software  tools  hide  the  message  into  the  image  data  by  decreasing  the  color 
depth  of  the  GIF  image  to  128,  64,  or  32  before  the  embedding  commences.  With  this  method, 
when  the  least  significant  bits  of  one,  two,  or  three  color  channels  are  perturbed,  the  total  number 
of  newly  created  colors  will  be  at  most  256.  [John98]  points  out,  however,  that  the  newly 


created  palettes  will  have  easily  deteetable  groups  of  close  colors.  It  is  thus  relatively  easy  to 
distinguish  images  with  and  without  embedded  messages. 

Still  other  software  tools  hide  the  message  into  the  image  data  in  the  form  of  parities.  Two 
examples  of  this  method  are  implemented  in  EzStego  and  in  Secure  Stego. 

EzStego  for  Indexed  Images 

One  of  the  most  popular  message  hiding  schemes  for  indexed  images  (i.e.,  GIF  files)  has  been 
proposed  by  Romana  Machado  [Mach].  In  her  method  called  EzStego,  the  palette  is  first  sorted 
by  luminance.  EzStego  embeds  the  message  in  a  binary  form  into  the  LSB  of  indices  (pixels) 
pointing  to  the  palette  colors.  Since  the  palette  is  sorted  aeeording  to  the  eolors,  typically 
invisible  ehanges  will  be  introduced  using  this  algorithm.  Here  are  the  steps: 

1 .  Find  the  index  of  the  pixel’s  RGB  eolor  in  the  sorted  palette. 

2.  Get  one  bit  from  the  binary  message  and  replaee  the  LSB  of  the  index. 

3.  Find  the  new  RGB  color  that  the  index  now  points  to  in  the  sorted  palette. 

4.  Find  the  index  of  the  new  RGB  eolor  in  the  original  palette. 

5.  Change  the  pixel  to  the  index  of  the  new  RGB  color. 
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Message:  0  1  1  0  0  1  0  101110101010001111 

-’/Kandomly  chosen  pixel  with  color  ^ 


Find  the  color  ®  in  the  luminance-sorted  palette 
Ci\ 


index  =  30  =  00011110 


H  000 1 1 1 1 0  Replace  the  LSB  of  the  index  to  \  ~^i 

i  color  Cl  with  the  message  bit  \  ^si|i|ili|gii|s 
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Figure  7.  Algorithm  for  EzStego. 

Message  recovery  is  simply  achieved  by  collecting  the  LSBs  of  all  indices  in  the  image  file.  Of 
course,  the  method  could  be  improved  by  injecting  message  bits  into  randomly  selected  pixels 
based  on  a  pseudo-random  number  generator  (PRNG)  seeded  with  a  secret  key. 


The  EzStego  algorithm  is  based  on  the  premise  that  close  colors  in  the  luminance-ordered  palette 
are  close  in  the  color  space.  However,  since  luminance  is  a  linear  combination  of  three  colors  R, 
G,  and  B,  occasionally  colors  with  similar  luminance  values  may  be  relatively  far  from  each 
other.  For  example,  colors  [6,98,233]  and  [233,6,98]  have  the  same  luminance  but  represent  two 
completely  different  colors.  Therefore,  on  occasion,  EzStego  will  introduce  large  changes  in 
color  during  message  embedding. 

Secure  Stego  for  Indexed  Images 

To  avoid  the  problem  inherent  to  EzStego,  [Frid99b]  proposes  to  hide  message  bits  into  the 
parity  bit  of  close  colors.  For  the  color  of  each  pixel  into  which  we  embed  message  bits,  we 
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search  the  closest  colors  in  the  palette  until  we  find  a  palette  entry  with  the  desired  parity  bit‘°. 
Since  the  parity  bits  of  palette  entries  corresponding  to  real  images  are  more  or  less  randomly 
distributed,  this  will  guarantee  that  we  will  never  have  to  depart  from  the  original  color  too 
much.  This  way,  we  avoid  the  problem  of  occasionally  making  large  changes  in  color.  [Frid99b] 
introduces  four  to  five  times  less  distortion  as  measured  by  root  mean  square  error  (RMSE)  in 
the  color  space  than  EzStego. 

Because  the  choice  of  the  parity  greatly  influenced  the  amount  of  distortion  introduced  into  the 
cover  image,  [Frid99e]  explored  the  possibility  of  generating  the  optimal  parity  assignment.  A 
rigorous  proof  is  given  that  shows  that  the  optimal  parity  assignment  introduces  --  on  the  average 
—  the  least  distortion  into  the  image  as  measured  by  the  RMSE.  [Frid99e]  also  extends  the 
optimal  parity  assignment  for  multiple-pixel  embedding  which  embeds  the  message  bit  into  q- 
tuples  of  pixels,  where  q>\.  Empirical  data  shows  that  when  q  =  2>,  one  introduces  half  of  the 
distortion  that  would  have  been  embedded  if  ^  =  1 . 

It  must  be  noted  that  RMSE  was  used  in  comparing  EzStego  with  Secure  Stego.  This  type  of 
measure  is  accepted  to  be  inadequate.  It  does  not  take  into  consideration  the  human  visual 
system  and  its  ability  to  recognize  the  distortion  due  to  message  embedding.  However,  in  the 
absence  of  an  acceptable  metric,  RMSE  is  used  as  the  de  facto  metric. 

3.2  Spectral  Techniques 

Spectral  techniques  are  more  robust  than  spatial  techniques  with  respect  to  image  processing 
operations.  However,  spectral  techniques  cannot  be  applied  to  all  types  of  images.  For  example, 
spectral  techniques  only  work  with  intensity  and  RGB  images  because  of  the  large  number  of 
available  intensities  or  colors.  Indexed  and  binary  images,  on  the  other  hand,  are  very  intolerant 
to  spectral  data  embedding  techniques  due  to  their  low  color  depth. 


In  [Frid99b],  the  parity  bit  of  the  color  (R,  G,  B)  was  defined  as  R+G+B  mod  2,  although  one 
can  arbitrarily  assign  parities  to  the  palette  color. 
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Figure  8  illustrates  the  general  procedure  for  frequency  domain  embedding.  Upon  applying  a 
frequency  transform  to  the  data,  a  perceptual  mask  is  computed  that  highlights  perceptually 
significant  regions  in  the  spectrum  that  can  support  the  watermark  without  affecting  perceptual 
fidelity.  The  data  to  be  embedded  is  then  inserted  into  these  regions. 


Embedded 
Signal,  w(k) 

Figure  8.  Data  Embedding  in  the  Frequency  Domain. 

In  principle,  any  frequency  domain  transform  can  be  used.  Of  the  different  possible  transforms, 
DCT,  FFT,  and  Wavelet  transforms  are  the  most  popular.  [Frid98a]  proposes  to  use  a  key- 
dependent,  random  orthogonal  transform  basis  instead  of  the  publicly  known  transforms.  The 
logic  behind  this  choice  is  to  make  directed  attacks  to  remove  the  embedded  data  much  more 
difficult  to  conduct  since  the  basis  functions  are  unknown  to  the  attacker.  It  is  arguable, 
however,  whether  this  increases  the  embedded  data’s  robustness  with  respect  to  blind  attacks. 

[Ramk99]  suggests  the  choice  of  the  transform  should  depend  on  the  robustness  needed.  For 
robustness  against  low  frequency  processing  scenarios,  the  discrete  cosine  and  wavelet 
transforms  are  appropriate  choices  since  they  are  better  energy  compaction  transforms.  But  if 
robustness  against  high  frequency  operations  is  required,  one  should  use  a  transform  with 
slightly  inferior  energy  compaction  properties  like  the  Hartley  transform. 

This  paper  presents  two  popular  spectral  data  embedding  techniques.  The  first  was  introduced  in 
[Cox95a];  the  second  presented  in  [Ruan98].  Since  [Cox95a]  and  [Ruan98]  are  based  on  direct 
spread  spectrum  encoding,  they  possess  both  the  advantages  and  disadvantages  of  spread 


30 


spectrum  techniques.  They  are  extremely  resistant  to  non-linear  distortions  of  amplitude  and  to 
additive  noise,  yet  are  extremely  intolerant  to  timing  errors  (loss  of  synchronization). 

Synchronization  is  of  utmost  important  during  watermark  extraction.  If  the  original  image  is 
present  (which  is  true  for  all  non-oblivious  techniques),  resynchronization  is  relatively  trivial 
[Ruan98].  If  not,  the  problem  becomes  extremely  difficult.  If  the  watermark  image  is  translated, 
rotated,  and  scaled,  then  synchronization  necessitates  a  search  over  a  four  dimensional  parameter 
space  (X-offset,  Y-offset,  angle  of  rotation,  and  scaling  factor).  The  search  space  grows  larger  if 
there  is  the  possibility  of  shear  and  of  changes  in  aspect  ratio.  As  pointed  out  earlier,  without 
having  access  to  the  original  image,  resistance  to  these  geometric  manipulations  (e.g., 
translation,  resizing,  rotation,  cropping,  and  warping)  is  still  an  unsolved  issue. 

Cox  et  al 

Cox  et  al  in  [Cox95a]  introduces  a  non-oblivious,  private  marking  scheme  that  may  employ 
perceptual  masking.  It  uses  a  spread  spectrum  approach  to  spread  one  message  bit  across  the 
entire  image.  It  embeds  a  single-bit  message  into  the  image. 

Watermark  Encoding 

Figure  9  shows  the  process  for  encoding  an  image  with  a  spread  spectrum  watermark.  Matlab 
code  of  the  Cox  et  al  data  embedding  encoder  is  listed  in  Section  8.  The  embedding  process 
consists  of  the  following: 

1 .  Generate  the  DCT  of  the  original  image. 

2.  Generate  the  perceptual  mask  that  identifies  the  significant  perceptual  components  of 
the  DCT.  [Cox95a,  95b,  96a,  96b]  indicated  that  there  are  many  different  methods 
for  determining  the  most  significant  perceptual  components  and  that  this  is  an  area  for 
further  research.  [Giro89,  Swan96c,  Podi98]  give  other  visual  models  to  determine 
the  most  significant  perceptual  components  into  which  to  embed  the  data.  The 
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method  [Cox95a]  opted  to  use  the  largest  1000  DCT  components  excluding  the  DC 
component. 

3.  Insert  the  watermark  into  the  perceptual  components  of  the  DCT.  Cox  et  al  discussed 
the  trades  in  choosing  the  length  of  the  watermark.  For  instance,  the  length  of  the 
watermark  affects  its  spread  among  the  significant  components  of  the  images.  In 
general,  the  magnitude  of  each  watermark  component  has  to  be  decreased  as  the 
length  of  the  watermark  increases  to  prevent  noticeable  image  degradation.  If  an 
image  is  especially  sensitive  to  changes  in  its  DCT  coefficients,  then  a  longer 
watermark  with  smaller  coefficients  may  be  used  without  significant  degradation  of 
the  watermark’s  robustness. 


Cox  et  al  proposed  using  a  watermark  consisting  of  a  sequence  of  real  numbers.  The 
length  of  the  sequence  is  defined  as  the  length  of  the  watermark,  in  this  case:  1 000. 
Many  variations  for  generating  a  watermark  are  possible.  For  example,  one  may  tune 
the  watermark  for  maximum  robustness  for  a  selected  set  of  image  corruption 
methods.  Each  watermark  real  number  value  was  chosen  independently  using  a 
random  standard  normal  distribution. 


Each  watermark  value  was  inserted  into  a  DCT  component  using  the  additive- 
multiplicative  equation: 

Vi’=  Vi(l  +  OCXi) 


where 

Xi  =  the  f*’  watermark  value 
Vi  =  the  original  DCT  element  value 
Vi’  =  the  modified  DCT  element  value 
a  =  the  strength  of  the  watermark 
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The  watermark  strength  was  set  to  a  =  0.1  in  Cox  et  al. 


4.  Generate  the  watermarked  image  by  taking  the  inverse  of  the  modified  DCT. 
Inverting  the  watermarked  DCT  produced  the  watermarked  image. 


Figure  9.  [Cox95a]  Watermark  Embedding  Algorithm. 


Watermark  Detection 

Cox  et  al  recovers  the  watermark  by  explicitly  computing  the  correlation  between  the  (noise 
corrupted)  watermark  recovered  from  the  image  with  a  set  of  perfect  watermarks  stored  in  a 
database.  This  is  a  very  robust  technique  for  watermark  recovery,  but  it  is  not  very  useful  in 
practice  because  of  the  need  for  access  to  the  database  of  perfect  watermarks  and  of  the  large 
amount  of  computation  required. 
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Figure  10  shows  the  watermark  detection  process  developed  by  Cox  et  al.  Matlab  code  of  the 
Cox  et  al  data  embedding  decoder  is  listed  in  Section  8.  It  consists  of  the  following  steps: 

1 .  Compute  the  DCT  of  the  recovered  and  of  the  original  unwatermarked  images. 

2.  Apply  the  perceptual  mask  to  extract  the  DCT  elements  that  contain  the  watermark. 
The  perceptual  mask  used  in  the  watermark  insertion  process  identifies  the  DCT 
locations  in  the  recovered  image  to  look  for  watermark  information.  The  same  DCT 
locations  are  also  extracted  from  the  original  images.  The  largest  1000  DCT 
components,  excluding  the  DC  component,  is  used  in  this  implementation. 

3.  Rescale  the  DCT  perceptual  elements  of  the  recovered  image.  The  extracted 
perceptual  elements  from  the  recovered  image  are  rescaled  to  match  the  vector  length 
of  the  perceptual  elements  from  the  original  image.  This  rescaling  is  an  addition  to 
Cox  et  al  watermark  detection  process.  It  was  introduced  by  Mike  Hafer  [Hafe97]. 

Without  the  rescaling  step,  [Hafe97]  found  that  the  watermark  detection  process  was 
sensitive  to  the  range  of  the  image  intensity  values.  Two  methods  were  investigated  to 
reduce  this  sensitivity.  The  first  method  is  to  rescale  the  intensity  values  of  image  for 
the  same  minimum  to  maximum  range  as  the  original  watermarked  image.  The 
second  method  rescales  the  length  of  the  DCT  perceptual  vector  of  the  recovered 
image  to  match  that  of  the  original  image.  The  rational  for  method  (2)  is  that  the 
intensity  range  effects  the  absolute  magnitudes  of  the  DCT  elements  but  not  their 
relative  magnitudes.  Therefore,  the  length  of  vector  V*  before  rescaling  is  affected  by 
intensity  range  variations,  but  its  direction  is  not. 

Rescaling  the  perceptual  DCT  elements  to  match  the  original  unwatermarked  image 
(instead  of  to  the  original  watermarked  image)  insures  that  a  watermark  bias  is  not 
applied  to  the  recovered  image.  This  bias  could  have  potentially  increased  the 


probability  of  a  false  positive.  Both  of  these  methods  eliminate  any  watermark 
detector  reading  variation  due  to  scaling  alone. 


4.  Extract  the  watermark  by  using  the  inverse  of  the  insertion  equation.  The  watermark 
is  extracted  by  using  the  inverse  of  the  equation  used  to  insert  the  watermark.  The 
insertion  equation  was 

Vi’=  Vi(l  +  OtXi) 

and  its  inverse  is 

Xi*  =  (vi*  -  Vi)/(avi) 


where, 

Xi*  =  the  i*  recovered  watermark  value 
Vi  =  the  original  DCT  element  value 
Vi*  =  the  recovered  DCT  element  value 
a  =  the  strength  of  the  watermark 
Recall,  that  the  watermark  strength  used  was  a  =  0. 1 . 


5.  Measure  the  similarity  of  the  extracted  watermark  against  the  original  watermark. 
The  similarity  of  the  original  water  mark  X  and  the  recovered  watermark  X*  was 
computed  by  Cox  et  al  in  their  experiments  using: 


sim(X,  X*)  =  (X*  •  X)/sqrt(X*  •  X*) 


One  can  normalize  sim(X,  X*)  was  normalized  for  a  maximum  value  of  one  by 
dividing  sim(X,  X*)  by  the  Euclidean  norm  of  the  original  watermark  vector,  X: 


sim_norm(X,  X*)  =  sim(X,  X*)/sqrt(X  •  X) 


Normalization  provides  the  advantage  of  knowing  the  maximum  watermark  detector 
reading  for  any  watermark.  This  simplifies  comparisons  when  different  watermarks 
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were  involved.  A  100%  watermark  hit  would  always  generate  a  1 .0  output  from  the 
detector  independent  of  the  element  values  of  the  original  watermark. 

The  detection  is  based  on  hypothesis  testing  and  the  existence  of  an  a  priori 
threshold.  If  sim(X,X*)  or  sim_norm(X,X*)  is  above  a  certain  threshold,  the 
watermark  is  said  to  be  present  in  the  image.  Otherwise,  the  watermark  is  considered 
absent. 


Watermark  Detector 
Reading 


Figure  10.  [Cox95a]  Watermark  Detection  Algorithm. 
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Refer  to  [Cox95a,  b,  96a,  b]  for  results  of  the  robustness  testing  for  this  technique. 

6  Ruanaidh 

An  elegant  method  for  coding  multiple  bits  into  a  spread  spectrum  signal  has  been  described  by 
6  Ruanaidh  [Ruan98]  (a  similar  technique  was  proposed  by  Piva  [Piva97]).  It  is  similar  to  the 
technique  proposed  by  Cox  et  al,  since  it  is  based  on  spread  spectrum  techniques.  However,  it 
does  not  require  access  to  a  database  of  watermarks  nor  does  it  need  prior  knowledge  of  which 
watermark  to  detect.  The  technique  introduced  by  [Ruan98]  is  an  oblivious  data  embedding 
scheme  that  may  perceptual  masking.  It  embeds  a  readable,  multiple-bit  message  into  the  image. 

The  watermark  is  inserted  by  adding  a  noise-like  signal  to  the  middle  frequencies  of  its  DCT. 

The  DCT  coefficients  are  converted  to  a  vector  and  the  middle  30%  (N^  frequencies)  is  chosen 
for  marking.  The  information  carried  by  the  watermark  consists  of  M  symbols  and  each  symbol 
Si  is  represented  using  r  bits,  1  <  5,  <  2^  For  each  /,  a  sequence  of  pseudo-random  numbers  of 
length  Nm-^2''  uniformly  distributed  in  [0,1]  is  generated.  For  each  symbol  a  new  sequence  of 
pseudo-random  numbers  is  generated.  Symbol  s  is  represented  using  the  segment 

consecutive  pseudo-random  numbers.  These  pseudo-random  vectors  may  be 

generated  by  any  "good"  pseudorandom  number  generator  (e.g.,  Gold  Codes,  Kasami  Codes,  m- 
sequences,  Legendre  sequences,  and  perfect  maps).  The  seed  for  the  PRNG  serves  as  the  secret 
key.  The  message  of  M  symbols  is  then  represented  as  a  summation 


S 


P 


The  spread  spectrum  signal  Sp  is  approximately  Gaussian  with  zero  mean  and  unit  standard 
deviation  even  for  moderate  values  of  M  (e.g.,  M =10).  The  signal  Sp  is  further  multiplied  by  a 
parameter  ^(watermark  strength  /  visibility)  and  added  to  the  middle  Nm  DCT  coefficients  dj. 
Again,  the  spatial  masking  model  of  Girod  [Giro89]  can  be  used  to  adjust  /so  that  the  double 
watermarked  image  is  perceptually  identical  to  the  original  image.  The  value  of  /=  13  works 
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well  for  most  images.  The  amplitude  of  the  combined  watermark  is  typically  in  the  range 
[-20,20]  with  an  average  rms  of  5  gray  levels.  In  [Ruan98],  the  watermark  was  repeatedly 
embedded  in  blocks  of  128x128  pixels.  Matlab  code  of  the  [Ruan98]  data  embedding  encoder  is 
listed  in  Section  9. 

The  detection  of  the  message  consisting  of  Af  symbols  proceeds  by  first  transforming  the  image 
using  a  DCT  and  extracting  the  middle  N„,  DCT  coefficients.  The  secret  key  is  used  to  generate 
M  pseudo-random  sequences  of  length  Nm+'2!^  needed  for  coding  the  message  symbols.  For  each 
sequence,  all  2''  segments  of  length  Nm  are  correlated  with  the  middle  Nm  DCT  coefficients.  The 
largest  value  of  the  correlation  determines  the  encoded  symbol.  Matlab  code  of  the  [Ruan98] 
data  embedding  encoder  is  listed  in  Section  9. 

This  watermarking  scheme  exhibits  very  impressive  robustness  properties  with  respect  to  many 
image  processing  operations.  Brightness/contrast  adjustment,  gamma  correction,  histogram 
operations,  dithering,  sharpening,  noise  adding,  and  high-pass  filters  leave  the  watermark  almost 
untouched.  The  watermark  is  also  fairly  robust  to  lossy  JPEG  compression.  Depending  on  the 
watermark  strength,  the  message  can  supposedly  be  extracted  untouched  after  JPEG  compression 
with  15%  quality.  Low  pass  filtering,  mosaic  filter,  and  median  rapidly  deteriorate  the  watermark 
especially  when  applied  iteratively  several  times. 

Refer  to  [Ruan98]  for  the  results  of  robustness  testing  for  this  technique. 

4  Limitations  of  Data  Embedding  Schemes 

Given  a  data  embedding  scheme,  an  unscrupulous  person  can  devise  a  distortion  that  removes  or 
destroys  the  embedded  data  without  severely  degrading  the  quality  of  the  image.  The  attacker 
has  the  advantage  here.  It  is  inconceivable  that  a  data  embedding  technique  is  designed  to  be 
robust  with  respect  to  all  possible  attacks.  This  section  presents  three  general  types  of  attacks: 
robustness  attacks,  presentation  attacks,  and  interpretation  attacks. 
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4.1  Robustness  attacks 

Robustness  attacks  aim  to  diminish  or  remove  embedded  data  from  the  stego  object.  A  basic 
attack  of  this  nature  would  be  to  lossy  compress  an  image  in  the  hope  that  the  resulting  image 
would  no  longer  have  the  embedded  data.  Pure  steganographic  techniques  that  trade-off 
robustness  for  undetectability  are  very  susceptible  to  common  signal  processing  operations. 

There  are  a  number  of  data  embedding  techniques  that  have  been  developed  for  digital 
watermarking  which  are  very  robust  with  respect  to  JPEG  compression,  additive  Gaussian  noise, 
low  pass  filtering,  scaling,  cropping,  brightness/contrast  enhancement,  global  rotation,  and 
cropping.  Even  so,  these  techniques  are  based  on  spread  spectrum  methodologies.  And  although 
spread  spectrum  signals  are  very  robust  to  amplitude  distortions  and  to  noise  adding,  they  are 
very  prone  to  synchronization  errors.  Synchronization  is  very  important  and  simple  systems  fail 
to  recover  this  synchronization  properly. 

4.2  Presentation  attacks 

Presentation  attacks  modify  the  content  of  the  stego  object  such  that  the  detector  cannot  find  the 
embedded  data.  This  paper  discussed  earlier  the  Stirmark  attack,  which  is  a  very  general  attack. 
Stirmark  introduces  sub-perceptual  distortions  that  render  the  embedded  data  unreadable. 
Another  general  presentation  attack  is  the  mosaic  attack. 

The  mosaic  attack  was  motivated  by  the  creatios  of  autonomous  agents  that  will  search  the 
Internet  for  images  that  contain  certain  embedded  information.  An  example  of  this  is 
MarcSpider  by  Digimarc  Corporation  [Digi99].  The  MarcSpider  service  searches  the  public 
Web  for  images  containing  digital  watermarks  and  produces  reports  on  where  and  when  such 
images  are  found.  Autonomous  agents  similar  to  the  MarcSpider  allows  Web  content 
developers,  photographers,  stock  photography  agencies  and  publishers  of  entertainment,  sports 
and  news  images  to  track  their  works  on  the  Web. 

The  mosaic  attack  consists  of  chopping  an  image  into  a  number  of  smaller  sub-images.  These 
smaller  images  are  tiled  on  a  web  page  such  that  a  web  browser  would  render  the  juxtaposed 
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sub-images  such  that  they  appear  identical  to  the  original  image.  [Peti98]  argues  that  this  attack 
is  quite  general;  all  marking  schemes  require  the  marked  image  to  have  some  minimal  size  (one 
cannot  securely  and  robustly  embed  a  meaningful  mark  in  just  one  pixel).  If  an  image  is  divided 
into  images  smaller  than  this  minimal  size,  the  embedded  data  can  no  longer  be  recovered. 

4.3  Interpretation  attacks 

Interpretation  attacks  devise  situations  which  prevent  the  assertion  of  the  embedded  data.  In  a 
digital  watermarking  architecture,  a  simple  interpretation  attack  will  be  to  insert  a  second 
watermark  into  a  previously  watermarked  image.  If  a  naive  embedding  scheme  was  used,  there 
would  be  no  intrinsic  way  of  determining  which  of  the  two  watermarks  was  added  first.  The 
invertibility  problem  presented  by  [Crav99]  —  which  is  also  known  as  the  IBM  attack  —  is  an 
example  of  an  interpretation  attack.  Timestamping  and  notarization,  as  they  are  performed  by 
the  U.S.  Copyright  Office  [USCo],  may  prevent  attacks  of  this  type. 

5  Applications 

There  are  four  clusters  of  applications  for  data  embedding.  One  application  uses  data  embedding 
techniques  to  convey  ownership  information  (source  authentication).  Another  application  uses 
digital  watermarking  techniques  to  verify  that  the  object  content  has  not  been  changed  (content 
authentication).  A  third  application  uses  steganography  to  convey  object-specific  information 
(e.g.,  captions)  to  a  community  of  willing  recipients.  The  last  application  is  the  use  of 
steganographic  techniques  to  convey  a  secret  message  while  concealing  the  very  presence  of  the 
secret  message. 

5.1  Source  Authentication 

In  the  business  environment,  applications  that  convey  ownership  information  are  often  desired 
by  organizations  that  own  the  copyrights  to  digital  media  objects  and  that  license  them.  [Mint99] 
describes  a  typical  image  application  as  follows:  the  content  owner  provides  an  image  that  is 
published  by  the  recipient;  in  exchange,  the  owner  receives  a  royalty  for  the  use  of  the  image.  A 
concern  of  the  owner  is  that  the  publisher  may,  intentionally  or  unintentionally,  neglect  to  pay 
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the  royalty.  To  deter  such  misappropriation,  the  owner  may  wish  to  place  ownership  information 
in  the  image. 

Traditionally,  handwritten  signatures  have  been  used  as  proof  of  authorship  of,  or  at  least 
agreement  with,  the  contents  of  a  document  because  of  the  following  assumptions  made  on  the 
signatures”  [Schn96]: 

1 .  The  signature  is  authentic.  The  signature  convinces  the  document’s  recipient  that  the 
signer  deliberately  signed  the  document. 

2.  The  signature  is  unforgeable.  The  signature  is  proof  that  the  signer,  and  no  one  else, 
deliberately  signed  the  document. 

3.  The  signature  is  not  reusable.  The  signature  is  part  of  the  document;  an  unscrupulous 
person  cannot  move  the  signature  to  a  different  document. 

4.  The  signed  document  is  unalterable.  After  the  document  is  signed,  it  cannot  be 
altered. 

5.  The  signature  cannot  be  repudiated.  The  signature  and  the  document  are  physical 
things.  The  signer  cannot  later  claim  that  he  or  she  did  not  sign  the  document. 

With  digital  documents,  we  would  like  to  use  a  concept  similar  to  handwritten  signatures. 
However,  there  are  problems  due  to  the  nature  of  digital  media.  First,  computer  files  are  trivial 
to  copy  —  it  is  easy  to  cut  and  paste  a  valid  signature  from  one  document  to  another.  The  mere 
presence  of  such  a  signature  means  nothing.  Second,  computer  files  are  easy  to  modify  after 
they  are  signed,  without  leaving  any  evidence  of  modification. 


”  [Schn96]  points  out  that  in  reality,  none  of  these  assumptions  is  completely  true.  Signatures 
can  be  forged;  signatures  can  be  lifted  from  one  piece  of  paper  and  moved  to  another;  and 
documents  can  be  altered  after  signing.  However,  Schneier  notes  we  are  willing  to  live  with 
these  problems  because  of  the  difficulty  in  cheating  and  the  risk  and  penalty  associated  with 
detection. 
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To  overcome  these  potential  weaknesses,  digital  signature  protocols  were  developed  [Schn96]. 
However,  a  weakness  of  using  digital  signatures  is  that  the  signature  is  separable  from  the 
document.  An  unscrupulous  individual  can  throw  away  the  original  signed  hash,  generate  his 
own  hash  with  his  own  private  key,  and  claim  ownership  to  the  document.  To  avoid  this 
situation,  data  embedding  techniques  can  be  used. 

Figure  1 1  illustrates  the  source  authentication  scenario  using  a  data  embedding  protocol.  The 
author  of  a  digital  image  wants  to  "sign"  the  image  so  that  no  one  else  can  attribute  the 
authorship  of  the  image  to  himself.  The  signature  can  be  appended  to  the  image  file,  be  visibly 
imprinted  on  the  image,  or  be  protected  by  a  digital  signature.  However,  these  signing  methods 
have  their  drawbacks.  If  the  signature  is  appended  to  the  image  file,  an  attacker  can  simply 
remove  or  replace  it  with  the  attacker’s  own  signature.  If  the  signature  is  visibly  imprinted  on  the 
image,  an  attacker  can  simply  crop  it  out.  Cryptographic  digital  signatures  offer  no  protection 
because  they  can  be  simply  replaced.  If  the  author’s  signature  is  embedded  into  the  image,  a 
recipient  can  determine  who  owns  the  image  by  detecting  the  embedded  signature.  In  addition, 
if  the  signature  is  embedded  into  the  image,  it  is  hoped  that  the  attacker  has  a  more  difficult  job 
of  removing  the  signature  without  severely  degrading  the  image. 
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Figure  11.  Source  Authentication. 

The  ownership  information  may  identify  either  the  owner  or  the  recipient  [Mint99].  If  the 
watermark  identifies  the  owner,  the  owner  might  subsequently  scan  suspect  published  material  to 
determine  whether  a  printed  image  contained  his  watermark;  the  owner  would  consider  its 
presence  to  be  evidence  of  his  ownership.  If  the  watermark  contained  the  recipient’s 
identification,  the  owner  might  subsequently  scan  the  published  material  to  determine  who 
received  the  material.  If  the  image  had  been  used  without  the  payment  of  royalties,  the  owner 
might  wish  to  cease  doing  business  with  the  recipient. 

In  military  applications,  instead  of  embedding  ownership  information,  one  can  embed  source 
information.  In  a  typical  image  application,  a  reconnaissance  sensor  can  tag  its  products  with  its 
ownership.  Since  the  information  is  embedded  into  the  image  data,  it  is  inseparable  fi'om  the 
image.  One  will  always  be  able  to  retrieve  information  pertaining  to  the  source  of  the  imagery. 
This  is  more  advantageous  than  having  this  type  of  information  stored  in  a  header  field  or  in  a 
separate  file;  header  information  can  always  be  overwritten;  links  to  the  separate  file  can  always 
be  broken. 
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For  a  data  embedding  scheme  for  source  authentication  to  be  effective,  the  embedded 
information  (i.e.,  the  watermark)  must  remain  in  the  media  object  and  must  be  reliably  detectable 
from  the  published  product.  This  is  a  considerable  challenge.  A  digital  image,  for  example,  can 
undergo  numerous  transformations  in  its  lifetime.  Sharpening,  contrast  enhancement,  color 
correction,  filtering,  and  compression  are  just  examples  of  image  manipulation  operations  an 
image  can  undergo.  A  watermark  must  survive  these  various  image  processing  operations.  In 
addition  to  simple  image  processing  operations,  a  watermark  must  survive  geometric  distortions 
(e.g.,  rotation,  cropping,  re-scaling).  Moreover,  it  must  also  be  taken  into  consideration  that 
there  is  an  economic  incentive  for  unscrupulous  people  to  remove  watermarks.  Thus,  an 
ownership  watermark  must  also  be  robust  with  respect  to  directed  attacks  against  the  watermark. 
Watermarks  that  withstand  intentional  or  imintentional  attacks  are  considered  robust  watermarks. 
Previous  works  are  [Cox95a],  [Hart97c],  [Kund97],  [Ruan98]. 

Aside  from  the  robustness  issue,  a  reliable  source  authentication  architecture  must  exist  to 
support  the  source  authentication  scheme.  [Adel99]  proposes  an  architecture  that  might  provide 
the  support  needed  for  a  source  authentication  /  copyright  protection  protocol. 

A  generic  copyright  protection  protocol  will  have  two  mechanisms:  a  registration  mechanism 
and  a  arbitration  mechanism.  The  registration  mechanism  (Figure  12)  will  accept  a  digital  object 
(Va)  and  a  digital  identity  (ID a)  from  a  principal,  say  Alice.  With  these  objects,  it  will 
determine  whether  the  digital  object  had  been  previously  registered.  If  the  digital  object  had 
been  registered,  the  registration  mechanism  will  abort.  If  not,  the  registration  mechanism  will 
issue  Alice  a  proof  of  ownership  and  the  watermarked  version  of  Va. 
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Figure  12.  Generic  Registration  Mechanism. 

In  an  ownership  dispute,  the  arbitration  mechanism  (Figure  1 3)  will  require  the  disputing  parties 
to  present  their  digital  identities  and  their  proofs  of  ownership,  as  well  as  the  digital  work  under 
dispute.  The  registration  center  will  have  to  determine  which  of  the  disputing  parties  is  the 
rightful  owner  of  the  digital  work.  The  registration  center  must  also  check  whether  the  digital 
work  belongs  to  a  party  not  involved  in  the  arbitration.  If  the  registration  mechanism 
competently  performed  its  task,  the  arbitration  mechanism  can  simply  verify  the  proofs  of 
ownership  presented  to  it  and  search  its  database  for  the  rightful  owner. 
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Figure  13.  Generic  Arbitration  Mechanism. 
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However,  the  registration  mechanism’s  responsibility  is  more  complex  than  presented  here.  In 
determining  whether  a  digital  work  has  been  previously  registered,  similarity  tests  must  be 
performed.  Is  the  digital  object  being  registered  similar  to  a  previously  registered  object?  The 
similarity  tests  must  consider  an  equivalence  relationship  and  not  an  equality  relationship.  For 
example,  the  similarity  test  should  not  be  deceived  by  scaling-rotation  attacks  or  by  brightness  / 
contrast  enhancements.  The  nature  of  the  equivalence  test  is  an  open  issue  and  is  one  of  the  most 
difficult  problems  in  this  application.  The  performance  of  the  registration  and  arbitration 
mechanism  must  also  be  taken  into  consideration.  The  time  it  would  take  to  search  the  database 
for  similar,  registered  objects  must  be  acceptable. 

The  registration  center  must  also  be  trusted  and  held  accountable  for  its  actions.  As  pointed  out 
by  [Adel99],  a  dishonest  center  could 

1 .  refuse  registration,  after  obtaining  the  original  work,  and  claim  that  a  similar  work 
has  already  been  registered, 

2.  issue  a  wrong  ownership  certificate,  i.e.,  a  certificate  not  suitable  for  ownership 
proofs, 

3.  collude  with  a  xxnscrupulous  party  and  issue  him  an  ownership  certificate  for  an 
previously  registered  object,  and 

4.  return  an  incorrect  result  during  an  ownership  dispute 

The  copyright  protection  architecture  is  similar  to  the  existing  United  States  Copyright  Office. 
However,  without  a  robust  and  comprehensive  similarity  test,  it  is  debatable  whether  such  an 
architecture  can  be  automated.  Nevertheless,  the  use  of  robust  digital  watermarks  can  still  be 
used  as  an  initial  check  for  ownership.  For  more  information  on  the  U.S.  Copyright  Office,  the 
reader  is  referred  to  http://lcweb.loc.gov/coDvright/. 

5.2  Content  Authentication 

The  second  application  area  uses  watermarks  to  determine  whether  a  media  object  has  been 
altered  since  some  earlier  time  when  it  was  watermarked.  Current  authentication  techniques  are 
based  on  cryptographic  principles  and  digital  signatures.  The  advantages  of  using  a  digital 
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signature  protocol  are  twofold.  First,  it  is  computationally  easy  to  compute  a  hash  value.  This 
minimizes  the  overhead  associated  with  signing  documents.  Second,  the  signature  is  sensitive  to 
any  modification  to  the  post-transmission  document.  Digital  signatures  are  good  for  complete 
authentication  —  the  data  considered  as  untouchable  [Lin99].  These  techniques  protect  every 
single  bit  of  the  content  and  do  not  allow  any  form  of  manipulation.  As  such,  to  be  considered 
authentic,  the  data  under  test  has  to  be  equal  to  the  original  one. 

In  real  applications,  this  may  not  be  practical.  In  several  situations,  digital  images  may  be  lossy 
compressed  and  may  be  digitally  enhanced  and  manipulated.  Moreover,  they  may  simply  be 
converted  from  one  graphical  file  format  to  another.  The  operations  do  not  affect  the  semantic 
meaning  of  the  image.  Because  the  significance  of  digital  images  is  based  on  their  content,  and 
not  on  their  bits,  manipulation  on  the  bits  that  do  not  change  the  meaning  of  the  content  should 
not  affect  an  image’s  authenticity.  On  the  other  hand,  manipulations  that  add  or  remove  features 
from  an  image  should  be  detected  and  possibly  localized  while  validating  the  remaining  areas  of 
the  image. 

Since  the  meaning  of  digital  images  is  based  on  its  content,  one  can  modify  the  image  bits  to 
embed  some  codes,  i.e.,  watermarks,  without  changing  the  meaning  of  its  content.  Because  the 
watermarks  are  embedded  in  the  data  content,  once  the  data  is  manipulated,  these  watermarks 
will  also  be  modified  such  that  the  authenticator  can  examine  them  to  verify  the  integrity  of  the 
data.  In  a  typical  image  application,  an  image  is  watermarked  at  the  time  it  is  loaded  into  a 
digital  library.  Later,  the  watermark  is  extracted.  If  the  extracted  watermark  matches  the 
inserted  watermark,  the  object  is  judged  to  be  unchanged;  if  it  does  not  match  the  inserted 
watermark,  the  image  is  judged  to  have  been  altered.  An  inspection  of  the  extracted  watermark 
and  its  difference  from  the  inserted  watermark  can  be  used  to  reveal  where  the  alterations  have 
occurred.  [Frid99c]  has  also  shown  that  it  is  possible  to  store  information  about  the  image  in  the 
image  itself  If  the  image  has  been  modified,  one  can  localize  where  the  image  had  been  altered 
and  reconstruct  the  pre-modified  image. 
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Adequate  content  authentication  schemes  must  be  able  to  distinguish  between  malicious  and 
non-malicious  attacks  on  the  image.  Malicious  attacks  are  attacks  that  modify  the  content  of  an 
image.  These  attacks  include  manipulating,  deleting,  or  adding  features  in  an  image.  Non- 
malicious  attacks  modify  the  pixel  values  of  an  image,  but  do  not  modify  its  contents.  Examples 
of  non-malicious  attacks  are  adjustment  of  brightness  or  contrast,  lossy  compression,  or  filtering. 
Research  conducted  using  data  embedding  techniques  for  content  authentication  use  fragile 
watermarking  schemes.  These  fragile  watermarking  schemes  are  robust  to  certain  image 
processing  operations,  but  will  be  significantly  altered  or  destroyed  if  the  image  is  altered. 
Previous  works  are  [Walt95],  [Lin97, 98a,  99],  [Wu98],  [Yeun97],  [Frid98d,  98e,  99c],  and 
[Bald99]. 


Original  Image  Tampered  Image 


Figure  14.  Content  Authentication  Scenario. 

Figure  14  illustrate  the  need  for  content  authentication  techniques.  With  the  advent  of  digital 
images,  the  concept  of  an  "original"  image  becomes  blurred.  Unlike  traditional  images  and  their 
chemical-based  processing,  digital  images  do  not  have  the  luxury  of  having  negatives  to  verify 
content  authenticity.  However,  if  the  digital  image  was  specially  prepared,  unauthorized 
modifications  to  an  image  can  be  detected.  Tamper  detection  seals  were  embedding  in  the  least 
significant  bits  of  the  original  image  in  Figure  14  in  accordance  with  the  scheme  proposed  in 
[Bald99].  Baldoza  et  al  extends  the  authentication  scheme  proposed  in  [Walt95]  by  subdividing 
the  entire  image  into  sub-blocks  and  ultimately  protecting  each  sub-block  by  introducing  the 
walk-dependent  checksum.  Because  the  original  image  is  divided  into  sub-images,  one  can 
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localize  where  the  image  had  been  altered.  Figure  15  demonstrates  the  capability  of  detecting 
and  localizing  xmauthorized  modifications  to  the  image. 


Figure  15.  Detection  of  Tampering  in  the  Content  Authentication  Scenario. 

Figure  16  illustrates  potential  instances  in  an  imagery  collection  and  dissemination  process 
wherein  the  integrity  of  an  image  can  be  compromised.  At  Point  A,  the  source  of  the  imagery 
can  be  altered  if  an  adversary  has  the  ability  to  intercept  communication  between  a  fnendly 
reconnaissance  platform  and  its  support  communication  relay.  The  hostile  imagery  platform  can 
either  degrade  the  original  imagery  or  substitute  it  for  a  downgraded  and  possibly  altered  image. 
At  Point  B,  attacks  can  be  taken  against  the  imagery  at  several  instances.  For  example,  the 
imagery  can  be  attacked  while  it  resides  on  the  database  prior  to  authorized  exploitation.  In 
addition,  the  imagery  can  be  inadvertently  modified  by  an  image  analyst.  In  either  case,  the 
intelligence  gathered  by  exploiting  an  altered  image  or  by  faulty  exploitation  may  have  grave 
consequences. 
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Figure  16.  Potential  Security  Breaches  in  the  Collection  and  Dissemination  Architecture. 

Cryptographic  techniques  can  be  used  to  safeguard  collected  imagery,  but  only  while  the  image 
remains  in  ciphertext  form.  Once  the  cryptographic  wrapper  is  removed,  the  images  are 
susceptible  to  malicious  and  inadvertent  modifications.  This  problem  was  discussed  earlier  in 
the  section  dealing  with  source  authentication  using  digital  signatures. 

The  weakness  of  using  this  cryptographic  protocol  is  not  that  the  adversary  can  throw  away  the 
original  signed  hash,  generate  his  own  hash  with  his  own  private  key,  and  claim  ownership  to  the 
image.  Instead,  the  weakness  is  that  once  the  image  has  been  authenticated,  the  signed  hash  may 
be  disassociated  with  the  image  —  through  negligence  or  through  malicious  intent.  If  this  occurs, 
subsequent  authentications  are  not  possible.  Any  modification  to  the  image  once  the  signed  hash 
is  disassociated  with  it  is  undetectable.  This  weakness  can  be  addressed  by  using  data 
embedding  techniques  to  embed  tamper-detection  seals  into  the  image.  Ideally,  these  seals  will 
break  when  the  image  has  been  altered. 

Several  different  schemes  that  use  data  embedding  techniques  can  be  used,  each  providing 
certain  advantages.  If  any  modification  to  the  image  is  to  be  detected,  fragile  embedding 


techniques  that  "break"  at  the  slightest  perturbations  should  be  used.  If  non-malicious 
modifications  should  not  flag  an  image  as  being  altered,  techniques  that  are  more  robust  should 
be  used.  Therefore,  images  that  had  undergone  processing  operations  such  as  lossy  compression, 
blurring,  sharpening,  and  contrast/brightness  enhancement  should  pass  the  authentication 
assessment.  However,  images  wherein  which  features  have  been  removed  (e.g.,  a  surface-to-air 
missile  site  had  been  cropped  out)  or  modified  (e.g.,  a  face  had  been  replaced)  should  fail  the 
authentication  process. 

One  image  processing  operation  that  should  be  considered  non-malicious  is  ortho-rectification. 
Ortho-rectification  is  a  complex  operation  involving  subtle  mathematical  calibrations  to 
represent  the  curved  surface  of  the  earth  as  a  flat  map.  Although  this  operation  does  not  change 
the  content  of  the  image,  the  non-linear  geometric  distortions  introduced  into  the  image  currently 
renders  any  current  data  embedding  technique  impotent.  Unless  the  non-linear  distortions  can  be 
reversed,  any  embedded  data  is  rendered  unreadable  by  this  operation.  A  data  embedding 
technique  that  is  robust  to  ortho-rectification  must  be  developed  before  data  can  be  reliably 
embedded  into  satellite  and  aerial  imagery^^. 

5.3  Captioning 

Captioning  applications  are  the  fnendliest  environment  for  data  embedding  techniques.  In 
captioning  applications,  both  the  content  owners  and  the  recipients  desire  that  the  information  be 
conveyed  -  there  is  no  economic  incentive  for  unscrupulous  people  to  remove  the  embedded 
data.  In  addition,  it  does  not  matter  whether  the  embedded  data  can  be  detected.  A  tj^ical 
image  application  has  a  digital  photo  agency  embedding  information  regarding  the  name  of  the 
owning  agency  and  an  image  identification  number.  When  the  image  is  published,  the  publisher 
knows  whom  to  contact  for  permission.  In  addition,  the  photographer  can  embed  aperture, 
shutter  speed,  and  subject  information  to  capture  the  eonditions  in  which  the  photo  was  taken. 


Incidentally,  any  data  embedding  teehnique  that  is  robust  with  respect  to  ortho-rectification 
will  also  be  robust  to  the  StirMark  attack  developed  by  F.  Petitcolas  [Peti98b].  This  is  the  "holy 
grail"  of  commercial  watermarking  techniques  ~  an  oblivious  data  embedding  technique  robust 
with  respect  to  geometric  deformations. 
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Traditional  captioning  techniques  for  digital  images  simply  attach  the  caption  to  the  image  via  a 
field  in  the  file  format  or  a  separate,  readable  file.  If  security  was  an  issue,  the  caption  is 
encrypted  before  it  is  attached  to  the  image.  The  main  weakness  to  the  simple  attachment  of  the 
information  is  that  the  link  between  the  image  and  the  caption  can  easily  be  severed.  In  certain 
applications,  this  loss  of  information  would  not  be  beneficial  for  the  eonsumer.  To  preclude  the 
loss  of  the  captioning  information,  one  is  motivated  to  embed  the  data  within  the  image  so  that 
they  become  inseparable. 

Captioning  applications  include  movie  dubbing  in  multiple  languages,  subtitles,  and  tracking  the 
use  of  the  data  (history  file).  For  example,  one  copy  of  a  movie  can  be  distributed  with  subtitles 
in  several  languages.  The  VCR,  DVD  player,  TV  set,  or  other  video  device  can  access  and 
decode  the  additional  text  (subtitles)  in  real  time  from  each  frame,  and  display  it  on  the  TV 
screen.  Although  this  could  be  arranged  by  appending  information  rather  than  invisibly 
embedding  it,  bandwidth  requirements  and  necessary  format  ehanges  may  not  allow  us  to  do  so. 

Figure  1 7  illustrates  the  captioning  application.  Information  regarding  the  image  is  embedded 
into  the  image.  This  information  is  available  later  for  anyone  to  retrieve. 
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Figure  17.  Captioning  Application. 

In  military  applications,  one  can  embed  metadata.  In  a  typical  image  applieation,  a 
recormaissanee  sensor  provides  an  image  that  will  be  exploited  by  the  image  analysts. 
Information  regarding  the  sensor  (e.g.,  sensor  type)  or  its  image  acquisition  parameters  (e.g., 
longitude  and  latitude,  slant  angle)  can  be  embedded  into  the  image  for  subsequent  retrieval. 
Again,  this  is  more  advantageous  than  having  this  type  of  information  stored  in  a  header  field  or 
in  a  separate  file:  header  information  ean  always  be  overwritten;  links  to  the  separate  file  ean 
always  be  broken.  A  possible  weakness  of  embedding  the  metadata  is  the  upper  limit  on 
eapacity  -  just  how  much  data  can  be  embedded  into  the  image.  In  addition,  there  is  a 
proeessing  penalty  associated  with  extraeting  the  data. 


In  the  captioning  application,  original  images  are  not  available  for  message  extraction.  Although 
the  reeipients  desire  to  extract  the  embedded  information,  some  robustness  is  required.  Captions 
should  survive  unintended,  non-malicious  attaeks  (such  as  compression),  but  they  are  not 
required  to  survive  directed  attacks  intended  to  remove  the  captions.  As  mentioned  earlier,  the 
consumer  is  not  motivated  to  remove  the  hidden  information  since  the  hidden  information  is 
beneficial  for  the  eonsumer.  Performance,  however,  may  be  an  issue  —  if  the  embedded 
information  must  be  recovered  in  real  time  (e.g.,  subtitles  in  a  movie),  fast  deteetion  is  neeessary. 
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5.4  Covert  Communications 

In  lieu  of  embedding  ownership  codes  or  captions  into  an  image,  one  can  embed  any  bitstream 
into  a  digital  image.  This  bitstream  would  be  retrievable  at  a  future  time  by  either  the  embedder 
or  an  intended  recipient.  A  typical  scenario  in  a  military  setting  would  be  a  covert  agent  in  a 
hostile  environment  embedding  secret  information  within  an  innocuous-looking  photograph. 

The  agent  will  then  transmit  the  photograph,  along  with  an  appropriate  innocent-looking  email 
message  over  the  Internet.  If  the  secret  message  was  embedded  properly,  an  email  message 
along  with  its  attached  photograph  will  not  arouse  any  suspicion.  Any  person  (or  program) 
scanning  message  traffic  would  only  see  the  stego  object  —  the  innocent-looking  picture  and  its 
message  —  and  not  the  secret  message.  Figure  18  demonstrates  this  application.  Attached  to  the 
email  is  a  group  photo  into  which  the  secret  message  had  been  embedded.  Only  with  the  correct 
stegano graphic  key  (and  possibly  a  separate  cryptographic  key)  will  one  be  able  to  extract  the 
embedded  message. 


Figure  18.  Covert  Communication  Application  -  Embedding  the  Secret  Message  in 

Innocent-Looking  Email  Messages. 
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The  advantage  steganographic  techniques  have  over  simple  cryptography  are  obvious. 

Encryption  simply  transforms  a  message  into  something  unintelligible.  Since  nothing  is  done  to 
hide  communication  channel,  the  message  traffic  between  the  transmitter  and  the  receiver 
remains  observable.  An  eavesdropper  can  see  the  message  traffic,  but  simply  cannot  understand 
it.  With  steganography,  the  secret  message  is  hidden  into  innocuous  messages.  An  eavesdropper 
can  read  the  innocuous  message  traffic,  but  cannot  detect  the  communication  of  the  secret 
messages.  For  a  secure  communication  channel,  it  is  advisable  to  use  both  cryptographic  and 
steganographic  techniques.  The  steganographic  wrapper  will  decrease  the  chances  of  the 
encrypted  message  traffic  from  being  detected;  the  cryptographic  wrapper  will  decrease  the 
probability  of  the  message  from  being  read  if  the  covert  channel  is  discovered. 

The  capability  of  embedding  messages  in  digital  images  has  its  dangers.  An  effective  method 
will  embed  data  into  the  images  without  introducing  statistically  detectable  artifacts.  It  is  not 
surprising  that  law  enforcement  agencies  are  attempting  to  develop  algorithms  that  will  detect 
the  presence  of  messages  in  multimedia  objects.  In  a  military  setting,  there  is  a  grave  possibility 
that  someone  may  have  concealed  classified  data  in  declassified  digital  imagery  [Kura92]. 

The  most  important  requirement  for  covert  communications  is  that  the  presence  of  the  hidden 
message  be  visually  imperceivable  and  statistically  undetectable.  Images  with  and  without  secret 
messages  should  appear  identical  to  all  possible  statistical  tests  that  can  be  carried  out.  In  the 
covert  communication  application,  detection  of  the  presence  of  a  hidden  message  is  equated  to 
the  failure  of  the  steganographic  technique,  even  if  detection  does  not  imply  the  ability  to  read 
the  message.  Undetectability  is  of  utmost  importance  especially  in  a  military  scenario  wherein 
the  presence  and  activities  of  covert  agents  must  be  kept  secret. 

For  the  embedded  message  to  be  visually  imperceivable  and  statistically  undetectable,  it  is  of 
paramount  importance  to  know  as  much  about  the  statistical  properties  of  the  source  from  which 
cover  images  are  being  drawn.  For  example,  if  the  images  are  scanned  photographs,  there  will 
be  stronger  correlation  in  the  horizontal  direction  than  in  the  perpendicular  direction  [Frid98c]. 
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[Fran99]  attempts  to  create  noise  models  of  digital  scanners  in  order  to  create  secure 
steganographic  techniques.  If  the  images  are  taken  using  a  digital  CCD  camera,  the  noise  will 
again  have  certain  specific  properties  induced  by  the  CCD  element  and  the  specific  data 
readout*^.  In  either  case,  the  data  hiding  scheme  must  respect  all  known  statistical  properties  of 
the  image  source  and  produce  images  that  cannot  be  distinguished  from  images  that  do  not 
contain  any  messages.  [Aura96]  offers  a  discussion  regarding  the  importance  of  respecting  the 
statistical  properties  of  the  image  source.  In  his  paper.  Aura  successfully  detected  messages 
embedded  using  LSB  encoding  using  statistical  measures.  In  addition,  [John98,  98b,  98c, 
West99]  give  a  variety  of  methods  of  detecting  data  embedded  in  images. 

Another  important  issue  is  the  capacity  of  the  communication  channel.  Capacity  is  also  related 
to  detectability.  Obviously,  the  longer  the  message,  the  larger  the  modification  of  the  cover 
image  and  the  higher  the  probability  that  the  modifications  can  be  statistically  detected.  It  is 
clear  that  one  can  embed  one  bit  of  information  into  one  frame  of  a  digital  video  without  any 
consideration  to  noise  models.  Such  communication  scheme  would  however  lead  to  an 
impractical  and  low  communication  bandwidth.  The  challenge  is  to  embed  as  much  information 
as  possible  while  staying  compatible  with  the  image  noise  model.  [Aura96]  determined  that 
replacing  five  percent  of  the  LSBs  with  pseudo-random  bits  does  not  significantly  change  the 
statistical  properties  of  an  image.  However,  increasing  the  capacity  such  that  100%  of  the  LSBs 
were  replaced  by  pseudo-random  bits  renders  the  message  detectable. 

The  last  important  requirement  is  that  it  must  not  be  possible  to  detect  the  hidden  message 
without  the  original  image.  If  the  cover  object  is  available,  it  is  trivial  to  detect  embedded 
messages.  Sometimes  it  may  be  possible  to  agree  on  certain  image  database  from  which  cover 
images  are  drawn  (without  repetition!)  but  this  obviously  limits  the  applicability  of  the 
technique. 


1  ^ 

In  a  recent  technical  exchange  with  Eastman  Kodak,  it  was  learned  that  images  taken  with 
digital  cameras  are  pre-processed  before  being  saved  onto  the  storage  device.  Therefore,  the 
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The  data  embedding  architecture  that  would  support  special  operation  forces  is  relatively  simple. 
In  fact,  the  Prisoner  s  Problem  epitomizes  the  scenario  covert  agents  routinely  find  themselves  in. 
The  two  principals  -  Alice  and  Bob  -  are  the  transmitter  and  receiver  on  the  communication 
channel.  Their  communication  channel  is  provided  using  email  through  the  Internet  and  is 
susceptible  to  eavesdropping  by  Willie,  the  warden.  If  Willie  detects  any  encrypted  or 
suspicious  message,  he  will  incarcerate  Alice  or  do  worse.  Therefore,  Alice  and  Bob  must  find 

some  way  of  concealing  their  covert  messages  in  an  innocuous-looking  exchange  of  public 
messages. 

Due  to  the  ubiquitous  nature  of  the  Internet  and  of  the  growing  popularity  of  digital  cameras,  a 
covert  operator  -  armed  with  a  computer  and  a  digital  camera  and  provided  with  Internet  access 
-  can  easily  blend  into  the  general  native  populace.  The  operator  can  then  transmit  messages 
over  the  Internet  either  through  email  (via  the  attachment  of  a  digital  image)  or  through  an 
Internet  site  (wherein  the  receiver  can  download  the  image).  Steganographic  techniques  for 
indexed  images  or  spectral  techniques  for  JPEG  images  can  provide  the  necessary  covert  channel 
of  communication.  Raw,  8-  or  24-bit  images  are  not  recommended  as  cover  objects  in  this  type 
of  scenario.  Transmitting  large  24-bit  image  files,  for  example,  instead  of  their  smaller  JPEG 
equivalents  may  arouse  suspicion.  Furthermore,  the  posting  raw  images  on  web  sites  are  rare,  as 
most  Internet  browsers  do  not  support  them. 


Undetectabihty  is  paramount.  The  uncovering  of  a  covert  agent  carries  great  implications 
ranging  from  diplomatic  embarrassment  to  the  loss  of  lives.  High  capacity  is  also  important.  An 
increase  in  the  volume  of  message  traffic  between  two  individuals,  in  itself,  may  not  be  cause  for 
suspicion.  However,  the  transmission  of  numerous  messages  quickly  depletes  the  database  of 
possible  cover  images.  As  discussed  earlier,  one  should  not  even  contemplate  using  the  same 
cover  object  twice.  Moreover,  the  original  cover  image  should  not  remain  available  after  the 


noise  model  of  these  devices  should  take  into  consideration  the  specific  properties  of  the  CCD 
element,  as  well  as  the  pre-processing  algorithms. 
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embedding  process.  If  one  of  these  requirements  are  violated,  the  warden  could  perform  a 
simple  differential  analysis  between  the  two  different  stego  objeets  and  determine  the  presence 
of  the  covert  message  (stego-  attack).  Figure  19  illustrates  the  architecture  of  this  attack. 
Differential  analysis  between  the  first  and  second  stego  object  can  be  performed  by  sunply 
subtracting  the  two  objects  pixel-wise.  If  one  pixel  in  the  first  image  is  not  equal  to  its 
coiresponding  pixel  in  the  other  object.  Willie  the  warden  will  be  alerted  to  the  extstence  of  a 

covert  coininunication  channel. 


If  Willie  detects  a  difference  between 
what  is  suppose  to  be  the  same  irnage, 
the  steganographic  technique  is 
defeated. 


Figure  19.  Stego*  Attack. 


Because  one  does  not  want  the  original  cover  object  to  be  available  the  creation  of  the  stego 
object,  it  would  be  advantageous  to  be  able  to  embed  the  message  at  the  source  (e.g.,  on  the 
digital  camera).  This  would  preclude  the  need  of  a  computer  to  perfonn  the  message  embeddmg. 
More  importantly,  it  would  also  eliminate  the  existence  of  the  original  cover  object.  If  the  digital 
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camera,  for  example,  were  to  fall  into  the  hands  of  the  warden,  the  warden  would  be  able  to  only 
view  the  stego  objects. 


Embedding  at  the  source,  however,  carries  with  it  potentially  dangerous  implications.  Take  the 
case  where  the  source  is  a  digital  camera.  One  obvious  requirement  is  that  the  steganographic 
technique  embedded  into  the  camera  works  on  and  outputs  the  same  image  formats  supported  by 
the  camera.  If  the  camera,  for  example,  produces  JPEG  and/or  (im)compressed  TIFF  images, 
then  the  steganographic  technique  must  have  as  input  and  output  JPEG  and/or  (un)compressed 
TIFF  images.  Steganographic  techniques  for  indexed  images  cannot  be  used  in  this  digital 
camera. 

More  importantly,  the  steganographic  technique  implemented  on  the  camera  must  not  violate  the 
noise  model  of  the  digital  camera.  If  the  warden  takes  possession  of  the  camera,  he  will  be  able 
to  construct  the  camera’s  noise  model  using  numerous  subsequent  photos  taken  by  the  camera. 
With  this  noise  model,  the  warden  will  be  able  to  test  whether  or  not  the  images  on  the  camera 
have  the  same  noise  characteristics.  A  single  image  that  violates  the  noise  model  will  signal  the 
detection  of  an  embedded  message.  It  does  not  matter  if  the  warden  can  read  the  embedded 
message  —  the  detection  of  the  embedded  message  defeats  the  steganographic  technique. 

Implementation  of  the  steganographic  sofiware/hardware  on  the  digital  camera  is  also  difficult. 

If  the  warden  has  access  to  the  digital  camera,  the  steganographic  software/hardware  will  have  to 
be  undetectable  has  well.  If  questioned,  the  owner  would  have  to  explain  why  his  camera  — 
which  is  supposed  to  be  commercially  available  —  has  dedicated  steganographic  modules.  This 
problem  is  compounded  if  a  cryptographic  system  is  also  embedded  in  the  camera.  If 
steganographic  hardware  and  software  are  commercially  available  and  are  recognized  as 
authorized  add-ons  to  the  camera,  the  owner  may  be  required  to  surrender  his  steganographic 
keys.  The  use  of  a  multi-level  steganographic  file  system,  similar  to  that  proposed  by  [McDo99], 
may  have  to  be  implemented. 


59 


Aside  from  implementation  concerns,  key  management  issues  have  to  be  addressed.  Storing  the 
steganographic  keys  in  the  camera’s  memory  would  make  the  warden’s  task  of  detecting  and 
extracting  the  embedded  messages  easier  once  he  gains  possession  of  the  camera.  Having  a  key 
pad  to  enter  the  key  will  likely  arouse  suspicion  as  to  the  existence  of  the  keypad.  Storing  the 
keys  on  flash  cards  that  can  then  be  loaded  and  unloaded  into  the  camera  has  to  be  carefully 
implemented.  The  warden  can  detect  the  keys  on  the  flash  card  just  as  easily  as  if  they  were 
stored  in  the  camera. 

6  Conclusion 

This  paper  introduced  data  embedding  schemes  for  covert  communications,  digital 
watermarking,  and  information  augmentation  applications  for  digital  images.  It  discussed  the 
data  embedding  framework  —  the  assumptions  implicitly  and  explicitly  made  on  the 
communication  channel,  the  implementation  issues  that  are  addressed  during  the  design  of  a  data 
embedding  scheme,  and  the  requirements  that  have  to  be  balanced  and  formalized.  For  covert 
communication  applications,  undetectability  is  paramount.  For  source  authentication 
applications,  robustness  to  a  variety  of  attacks  is  critical.  For  content  authentication  applications, 
one  must  be  able  to  distinguish  between  malicious  and  non-malicious  modifications.  In  a 
captioning  application,  capacity  is  maximized. 

The  paper  also  presented  several  examples  of  data  embedding  techniques  for  digital  images. 
Spatial  techniques  are  very  sensitive  to  simple  noise  adding  and  filtering.  Spectral  techniques, 
based  on  spread  spectrum  techniques,  are  susceptible  to  desynchronization  attacks.  Robustness 
to  geometric  deformations  is  still  an  open  issue  and  a  solution  must  be  found  before 
watermarking  techniques  are  successfully  applied  to  image  copyright  protection.  Limitations  of 
these  techniques  are  categorized  into  robustness  attacks,  presentation  attacks,  and  interpretation 
attacks. 

Several  application  scenarios  are  presented  and  the  maimer  in  which  the  data  embedding 
framework  is  typically  tailored  and  applied  to  them  is  proposed. 
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8  Appendix  A.  Matlab  code  for  Cox  et  al  Data  Embedding  Encoder  and  Decoder. 

The  Matlab  code  for  the  Cox  et  al  data  embedding  scheme  was  provided  by  Michael  Hafer 
[Hafe97]  and  has  been  divided  into  several  m-files: 


Filename 

Description 

wm_example.m 

Ties  the  other  three  supporting  m-  files  into  a  coherent 

architecture 

insertWaterMark.m 

Inserts  a  watermark  into  a  specified  image 

extractW  aterMark.m 

Extracts  a  watermark  fi-om  a  specified  image 

compareW  aterMarks 

Performs  the  similarity  function  needed  to  compare 

.m 

watermarks 

8.1  wm_example.m 

%  Filename:  wm_example.m 
%  Author:  Mike  Hafer,  12/98 

%  The  following  code  shows  an  example  of  the  watermarking  and 
%  extraction  process 

im_org  =  pgmRead( 'einstein.pgm' ) ; 

%  Create  a  random  watermark  of  length  1000 
waterMark  =  randndOOO,  1)  ; 
wmStrength  =  0.1; 

%  Create  watermarked  version  of  Einstein  image 

[im_WM,  wind]  =  insertWaterMark ( im_org,  waterMark,  wmStrength); 

%  Generate  recovered  (corrupted)  version  of  watermarked  Einstein  image 
im_recovered  =  im_WM; 
im_recovered (90 ,  20)  =  500; 

%  Extract  watermark  from  recovered  image 

wm_ext  =  extractWaterMark(im_org,  im_recovered,  wind,  wmStrength) ; 

%  Get  watermark  detector  reading  for  the  recovered  image 
similarity  =  compareWaterMarks (waterMark,  wm_ext) ; 
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8.2  insertWatermark.in 


%  Filename:  insertWaterMark.m 
%  Author:  Mike  Hafer,  12/98 

%  [im_WM,  VJMind]  =  insertWaterMark(im,  WaterMark,  strengthWM) 

% 

%  Inserts  a  watermark  into  an  image  to  produce  a  water  marked 
%  image.  This  is  a  modified  version  of  insertWaterMarkOrg . 

%  This  version  excludes  the  pure  vertical  and  horizontal  DCT 
%  elements  from  the  perceptual  mask. 

% 

%  im:  Image  to  be  water  marked,  D 
% 

%  WaterMark:  Water  mark  to  add  to  image,  X 
% 

%  StrengthWM:  Water  mark  strength 
% 

%  im_WM:  Water  marked  image,  D' 

% 

%  WMind:  Water  mark  indices  for  V' 

% 

function  [im_WM,  WMind]  =  insertWaterMark{im,  WaterMark,  strengthWM) 
lengthWM  =  length (WaterMark) ; 

%  Determine  the  perceptually  significant  elements,  V,  of  the 

%  image's  DCT.  This  is  done  by  finding  the  indices  corresponding 

%  to  the  largest  absolute  DCT  coefficient  values  excluding 

%  the  pure  vertical  and  horizontal  DCT  coefficients.  The  number  of 

%  indices  equals  "lengthWM" 

im_DCT  =  dct2(im); 

im_DCT_mod  =  im_DCT; 

im_DCT_mod ( : , 1 )  =  zeros (size (im_DCT,  1),  1);  %  Exclude  pure  vertical  coefficients  of  DCT 

im_DCT_mod ( 1 , : )  =  zerosd,  size{im_DCT,  2));  %  Exclude  pure  horizontal  coef  of  DCT 

im_DCT_vec  =  im_DCT_mod  ( : )  ,- 

sorted_DCT  =  f lipud ( sort ( abs ( im_DCT_vec ) ) ) ; 

lower_bound  =  sorted_DCT ( lengthWM) ; 

indl  =  find ( abs (im_DCT_vec)  >  lower_bound) ; 

ind2  =  find ( abs (im_DCT_vec)  ==  lower_bound) ; 

WMind  =  [indl;  ind2 ( 1 :( lengthWM  -  length{indl) ) ) ] ;  %  DCT  indices  for  V  &  V' 

%  Generate  water  marked  DCT  element,  V' : 
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%  V'  =  V*(l  +  strengthWM*X) 
im_DCT_WM  =  iin_DCT; 

V  =  iin_DCT  (WMind)  ; 

V_prime  =  V.*(l  +  strengthWM*WaterMark) ; 

%  Generate  DOT  for  water  marked  image. 
im_DCT_WM (WMind)  =  V_prime; 

%  Generate  the  water  marked  image,  D' 
im_WM  =  idct2 (im_DCT_WM) ; 


8.3  extractWaterMark.m 

%  Filename:  extractWaterMark.m 
%  Author:  Mike  Hafer,  12/98 

%  [WaterMark]  =  extractWaterMark(imOrg,  imWM,  WMind,  strengthWM) ; 

% 

%  Extracts  the  water  mark  from  an  image.  This  version 
%  rescales  the  DCT  perceptual  elements,  V*,  from  the  recovered  image 
%  to  match  the  norm  of  orginal  image's  DCT  perceptual 
%  elements,  V. 

% 

%  imOrg:  Original  unwatermarked  image,  D 
% 

%  imWM:  Recovered  watermarked  image,  D* 

% 

%  WMind:  Watermark  DCT  indices  for  V  and  V* 

% 

%  StrengthWM:  Watermark  strength 
% 

%  WaterMark:  Extracted  watermark,  X* 

function  [WaterMark]  =  extractWaterMark(imOrg,  imWM,  WMind,  strengthWM) 

%  Extract  the  DCT  perceptual  elements,  V,  from  the  original  image,  D 
DCT_org  =  dct2 ( imOrg) ; 

V_org  =  DCT_org (WMind) ; 

%  Extract  the  DCT  perceptual  elements,  V*,  from  the  recovered  image,  D* 
DCT_WM  =  dct2(imWM); 

V_WM  =  DCT_WM (WMind) ; 

%  Rescale  the  DCT  perceptual  elements,  V*,  from  the  recovered  image 
V_WM  =  V_WM  *  norm (V_org) /norm (V_WM); 
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%  Extract  the  watermark  using  the  equation: 

%  X*  =  (V*  -  V) / (V*strengthWM) 

WaterMark  =  (V_WM  -  V_org)  .  /  (V_org.  *strengthWM) 


8.4  compareWaterMarks.m 

%  Filename:  compareWaterMarks.m 
%  Author:  Mike  Hafer,  12/98 

%  [similarity]  =  compareWaterMarks (WM_org,  WM_extracted) ; 

% 

%  Evaluates  the  similarity  of  watermarks 
% 

%  WM_org:  Original  watermark,  X 
% 

%  WM_extractecl:  Extracted  watermark,  X* 

% 

%  similarity:  Similarity  reading; 

%  Value  of  1.0  indicates  an  exact  match 

% 

function  [similarity]  =  compareWaterMarks (WM_org,  WM_extracted) ; 
similarity  =  WM_extracted' *WM_org/norm(WM_extracted) /norm(WM_org) 
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9  Appendix  B.  Matlab  code  for  [Ruan98]  Data  Embedding  Encoder  and  Decoder. 

The  Matlab  code  for  the  [Ruan98]  data  embedding  scheme  was  provided  by  Jiri  Fridrich  and  has 
been  divided  into  several  m- files: 


Filename 

Description 

PivaTest.m 

Ties  the  other  three  supporting  m-  files  into  a  coherent 

architecture 

PivaWmk.m 

Inserts  a  watermark  into  a  specified  image 

PivaDetect.m 

Extracts  a  watermark  from  a  specified  image 

compare.m 

Performs  a  [Giro89]  comparison  between  two  specified 
images  in  order  to  compute  the  percentage  of  pixels 

with  visible  differences 

genMsg.m 

Generates  a  key-based  message 

localstd.m 

Calculates  the  local  standard  deviation  for  a  matrix. 

getindexy.m 

Returns  vector  of  indices  of  the  first  D  diagonally 

chosen  matrix  bins  following  DO  omitted  bins 

9.1  PivaTest.m 

function  PivaTest  (image,  output_image ,  L,  M,  alpha,  seed  ) ;' 

%  Watermark  Image 

[Xw,  betabar,  alphabar]  =PivaWmk (image,  output_iinage ,  L,  M,  alpha,  seed),- 

%  Compare  Original  with  Watermarked  Image 
[ X , Map ] =bmpread ( image ) ; 

[Xw,Map]  =bmpread  (output,_image)  ,- 

[pixels ,percent_bad] =compare (X, Xw, 0 . 003 ) ; 

fprintf  (' - >Percent  Change  Between  Image  and  Marked  Image  =  %f\n',  percent_bad)  ,- 

%  Detect  Watermark 

[ z, Sz, Tro] =PivaDetect (output_image,  L,  M,  alphabar,  seed) ; 
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9.2  PivaWink.in 

function  [Xw,betabar, alphabar] =PivaWmk(image,output_image,L,M, alpha, seed) ; 

%  Watermarlcing  method  by  Piva  from  A.  Piva,  M.  Bami,  F.  Bartolini, 

%  V.  Cappellini,  "DCT-based  watermark  recovering  without  resorting 
%  to  the  uncorrupted  original  image",  preprint,  1997. 

% 

%  Adjustment  for  visibility  is  done  using  simple  localized  std 
%  This  function  uses  localstd.m  for  calculating  localized  std 
%  and  getindexy.m  for  selecting  DCT  coeffs  in  a  zig-zag  manner 
% 

%  Inputs : 

%  image  Input  image  to  be  watermarked  (in  BMP  format) 

%  L  First  L  DCT  coefficients  will  be  skipped 

%  M  The  next  M  DCT  coefficients  will  be  modified 

%  alpha  Watermark  strength/visibility 

%  seed  Seed  for  a  PRNG 

% 

%  Output:  Watermarked  image  Xw 

%  Typical  usage:  Xw=pivalnew( 'pep512.bmp' ,25000, 16000, 2, seed) ; 

% 

%  Piva  recommends  using  L=25000,  M=16000,  and  alphabar=0.2,  alpha=alphabar/betabar 
%  The  visibility  of  the  watermark  stays  below  an  acceptable  level 
%  (according  to  spatial  masking  model  of  Girod) .  To  test  that,  type: 

%  compare{'pep512. bmp', 'water. bmp', 0.003)  right  after  running  pival.m. 

[X,Map] =bmpread( image) ; 

[Nl,N2]=size(X) ; 
original=X; 

rs=getindexy(Nl,N2,L,M) ; 

DX=dct2 (X) ; 

DXW=DX; 

%  Watermarking-beginning 

randn ( ' state ' , seed) ; 
eta=randn (M,l); 

DXw(rs)=DX(rs)+alpha*(eta.*abs(DX(rs) ) ) ; 

Xw=idct2 (DXw) ; 

Xw(find(Xw>256) )=256;  %  Take  care  of  boundary  effects 

Xw( find (Xw<l) ) =1 ; 


%  Diagonal  indices  of  the  first  M 
%  except  the  first  L  DCT  bins 
%  Calculate  the  2D  DCT  transform  of  X 
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%  Adjustment  for  visibility 


K=9; 

s=localstd (X, K) ; 
beta=s/max (max (s) ) ; 
Xw=beta. *Xw+ (1-beta) .*X; 


%  Kernel  size  for  calculating  local  std 
%  Matrix  of  local  stds 
%  Correction  factor  beta 

%  Adjustment  for  visibility  {beta~l  emphasizes  the 
%  watermarked  image,  beta~0  emphasizes  the  original) 


%  Show  the  original  image 

figure (1) ; imshow (original , Map) ; 
title (' Original  image'); 


%  Show  the  watermarked  image 
figure(2) ; imshow (Xw, Map) ; 
title ( 'Watermarked  image' ) ; 

bmpwrite(Xw,Map,output_image) ;  %  Store  watermarked  image  to  disk  as  'water.bmp' 

%  Information  about  the  energy  of  the  watermark 
df =Xw-original ; 
powerofwm=mean2 (df . "2 ) ; 

fprintf('  The  average  watermark  power  is:%f\n',  powerofwm) ; 

fprintf('  %d  <  Watermark  <  %d\n' ,  floor (min(min(df) )) ,  floor (max (max (df) ))) ; 

%  Control  output  (see  the  paper  by  Piva) 
betabar=mean2 (beta) ; 

alphabar=alpha*betabar;  %  this  alphabar  become  constant  in  detections! 


9.3  PivaDetect.m 

function  [ z, Sz, Tro] =PivaDetect ( image, L, M, alphabar, seed) ; 

%  Watermarking  method  by  Piva  from  A.  Piva,  M.  Barni,  F.  Bartolini, 
%  V.  Cappellini,  "DCT-based  watermark  recovering  without  resorting 
%  to  the  uncorrupted  original  image",  preprint,  1997. 


%  image  Watermarked,  attacked  image 

%  L  First  L  DCT  coefficients  will  be  skipped 

%  M  The  next  M  DCT  coefficients  will  be  modified 

%  alphabar  Watermark  strength  -  for  calculation  Sz  threshold 
%  seed  Seed  for  a  PRNG 

%  Typical  usage:  [z  Sz  Tro] =detlnew(image, 25000, 16000, 0 .2,  seed) ; 

%  for  image  size  other  than  512x512  L=25000*N1/512*N2/512  M=16000*N1/512*N2/512 


[X , Map ] =bmpread ( image ) ; 
[Nl,N2]=size(X) ; 
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original=X; 
DX=dct2 (X)  ; 


%  Calculate  the  2D  DCT  transform  of  X 


%  Detection-beginning 

rs=getindexy(Nl,N2,L,M) ;  %  Diagonal  indices  of  the  first  M 

%  except  the  first  L  DCT  bins 

randn ( ' state ' , seed) ; 
eta=randn (M, 1) ; 

Sz=sum(abs (DX(rs) ) ) ;  %  Image  dependent  threshold 

Sz=Sz*alphabar/ (3*M) ; 
z=sum(DX(rs) .*eta)  ; 
z=z/M; 

stdt=std(DX(rs) ) ;  %  An  alternative  threshold  follows 

Tro=3.3*sgrt(2/M*stdt*stdt) ;%  from  "Threshold  Selection  ..."  Piva's  paper 

9.4  compare.m 

function  [pixels, percent_badl =compare (imagel, image2,Th) ; 

%  This  function  is  used  to  find  out  if  differences  between  two  images  are  visible. 

%  The  model  used  is  the  spatial  masking  model  described  by  Girod.  The  inputs  are: 

%  two  images  in  BMP  format,  and  a  value  of  the  threshold  Th  for  the  saturation 
%  signal.  Based  on  computer  experiments  using  threshold. m  this  value  should  be  about  0.003. 
%  The  function  decides  whether  or  not  the  two  images  are  perceptually  identical. 

%  The  areas  containing  visible  differences  are  shown  in  black. 

% 

%  A  simplifying  assumption  of  a  constant,  homogenous  JND  in  the  saturation  signal  is  used 
%  to  invert  Girod's  spatial  masking  model.  For  more  details,  consult  Girod's  paper. 

% 

%  XI  ....  imagel 
%  X2  ....  image2 
%  Th  ....  threshold 

%  Typical  usage:  compare {' len256 .bmp' , 'water .bmp' , 0 . 003 ) ; 

% [XI , Mapl ] =bmpread ( imagel ) ; 

,.Xl=imagel; 

[Ml,Nl]=size(Xl) ; 

% [X2 , Map2 ] =bmpread ( image2 ) ; 

X2=image2 ; 

[M2,N2]=size{X2) ; 

if  N1~=N2  I  M1~=M2 
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disp('**  Warning:  **'); 

disp ( ' Images  have  different  dimensions . ' ) ; 
disp ( 'Truncating  dimensions 
end 


N=min{Nl,N2) ; 
M=min(Ml,M2) ; 


%  Viewing  characteristics 


v=4; 

Alpha= (180/pi) *2*atan(l/(2*v) ) ; 
Ppd=  (sqrt  (M''2+N''2 )  /Alpha)  /60; 


%  Viewing  distance  of  v  image  diagonals 
%  Angle  subtended  by  the  image  diagonal  (in  degrees) 
%  Ppd  =  pixels  per  arc  minute 


%  Parameters  of  the  spatial  masking  model  (according  to  Girod's  paper) 


Lmon=0 .00035; Smon=15 ;G=2 . 2 ; 
Ksat=8 ; 

Lad=  7 ; 

Sinh=8 ; 

Savg=13 ; 


%  Monitor  characteristics  provided  it  complies  with  CCIR 
%  Saturation  coefficient 
%  Level  of  ambient  illvunination 
%  Spatial  inhibition  spread  in  arc  minutes 
%  Saturation  averaging  spread  in  arc  minutes 


%  Spatial  masking 

L=Lmon*power (Xl+Smon, G) ;  %  Conversion  from  grayscale  to  screen  luminance 

wl=G*Lmon*power(Xl+Smon,G-l) ;  %  Factor  wl  for  the  linearized  model 

dL=wl.*(X2-Xl) ; 

h=fspecial ( 'gaussian' , [10  10] , Sinh*Ppd) ;  %  Gaussian  filter  10x10  with  sigma=Sinh  [arcmin] 

Linh=Lad+conv2 (L,h, 'same' ) ;  %  Inhibited  luminance  Linh  (see  Girod's  paper) 

w2=ones (M,N) . / (Linh+Ksat*max(2eros (M,N) ,L-Linh) ) ;  %  Factor  w2  for  the  linearized  model 

h=fspecial ( 'gaussian' , [10  10] , Savg*Ppd) ;  %  Averaging  Gaussian  kernel  with  sigma=Savg  [arcmin] 

dcsat=dL. *w2 ;  %  Increment  in  fovea  saturation 

dcsat2=dcsat . *dcsat;  %  Square  of  the  saturation  increment 

dS=filter2 (h,dcsat2) ;  %  Averaging  the  saturation  increment  using  h 

%  Determining  the  pixels  with  visible  changes 


bad_pixels=f ind (dS>Th) ; 
D=ones (M, N) ; 

D (bad_pixels) =0; 


%  Show  image 1  and  2 

subplot (1,3,1) ; imshow (XI , Mapl ) ; 
title ( ' Imagel ' ) ; 
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subplot (1,3,2) ; imshow(X2 ,Map2) ; 
title ( ' Image2 ' ) ; 

%  Show  the  bad  pixels  in  black,  the  rest  in  white 

subplot (1,3,3) ; 
imshow(D) ; 

title ('Black  pixels  =  visible  changes'); 
bad=length(bad_pixels) ; 

fprintf('  %d  pixels  (%f  percent)  have  visible  changes \n ' ,  bad,  bad*100/ (M*N) ) 

percent_bad  =  bad*100/ (M*N) ; 
pixels=length{bad_pixels) ; 


9.5  genMsg.m 

function  message=genMsg (nuitiberOf Symbols)  ; 

msg='X' ; 

rand  ('state',  13); 
for  count=l : (numberOfSymbols+1) 
i f  ( rem ( rand ( 1 ) , 2 )  >  0.5) 
msg  =  strcat  (msg,  '1'); 
else 

msg  =  strcat  (msg,  '0'); 

end 

end 

message  =  msg (2 : length (msg) -1); 


9.6  localstd.m 

function  s=localstd (X, K) ; 

%  This  function  calculates  the  local  standard  deviation  for  matrix  X. 

%  The  standard  deviation  (std)  is  evaluated  for  a  square  region  KxK 
%  pixels  surrounding  each  pixel.  At  the  boundary,  the  matrix  is  NOT 
%  padded.  Instead,  the  std  is  calculated  from  available  pixels  only. 

%  If  K  is  not  an  odd  integer,  it  is  floored  to  the  closest  odd  integer. 
% 

%  Input:  MxN  matrix  X 

%  K  size  of  the  square  region  for  calculating  std 

%  Output:  s  local  std  calculated  for  KxK  regions 
%  Typical  use:  s=localstd (X, 3 ) ; 
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[M  N] =size (X) ; 


if  inod(K,2)~=l  %  K  must  be  an  odd  number 

K=K+1; 
end 


kern=ones (K, K) ; 

xl=conv2 (X, kern, ' same' ) ;  % 
x2=conv2 (X . *X, kern, ' same' ) ;  % 
R=conv2 (ones (M,N) , kern, ' same' ) ; 
s=sqrt {x2 . /R- (xl . /R) . "2 ) ;  % 


%  Kernel  for  calculating  std 
Local  sums 

Local  cjuadratic  stims 

%  Number  of  matrix  elements  in  each  square  region 
Local  standard  deviation 


%  An  old  approach  using  blockprocessing  -  it  cannot  handle  matrices  larger  than  500x500 
%  and  incorrectly  calculates  std  at  the  boundaries 
%  W=zeros (M+2,N+2) ; 

%  W{2 :M+1, 2 :N+1) =X; 

%  z=zeros {M,N) ; 

%  z ( : ) =std (im2col (W, size (kern) , 'sliding' ) , 1) ; 


9.7  getindexy.m 

function  [ind, DIAG] =getindexy (M,N, DO , D) ; 

%  This  function  returns  vector  of  indices  of  the  first  D  diagonally  chosen 
%  matrix  bins  following  DO  omitted  bins 
%  MxN  matrix 

%  Scanning  order  for  4x5: 

%  1  3  6  10  14  18 

%  2  5  9  13  17  21 

%  4  8  12  16  20  23 

%  7  11  15  19  22  24 

%  Typical  usage:  ind=getindexy(512, 512, 25000, 16000) ; 

%  The  second  output,  DIAG  is  usually  not  needed 

DIAG=(1:M) '*ones(l,N)  +  ones (M, 1 ) * ( 1 :N) ;  %  row+col  matrix 

%  2  3  4  5  6  7 

%  3  4  5  6  7  8 

%  4  5  6  7  8  9 

%  5  6  7  8  9  10 

DIAG= (DIAG-2) . * (DIAG-1) /2  +  ones (M, 1 ) * ( 1 :N) ; 

Qo= ( (1 :M) ' *ones (1,N) ) + (ones (M, 1) * (1 :N) ) ; 

if  M==N 
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%  correction  for  the  lower  down  corner  part  of  the  matrix 


P=Qo- (M+1) ; 

P(P<0)=0; 

Z=P. *P; 

DIAG=DIAG-Z; 
else 

mi=min(M,N);  %  correction  for  the  center  paralelogram  part  of  the  matrix 

P=Qo- (mi+1 ) ; 

P(P<0) =0; 

Z=P.*(P+1) /2; 

DIAG=DIAG-Z; 

ma=max(M,N) ;  %  correction  for  the  lower  down  comer  part  of  the  matrix 

P=Qo- (ma+1) ; 

P(P<0)=0; 

Z=P.*(P+1) /2; 

DIAG=DIAG-Z; 

p=Qo-(N+l);  %  correction  for  “N+“  part  of  the  matrix 

P(P<0) =0; 

DIAG=DIAG+P,- 

end 

ind=find(DIAG<=D+DO  &  DIAG>D0) ; 
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